2024-05-11T12:01:38.383 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-11T12:01:38.383 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} --archive /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358 --owner scheduled_rishabh@teuthology --verbose --name rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi -- /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/orig.config.yaml 2024-05-11T12:01:38.438 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T12:01:38.534 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358 branch: wip-rishabh-testing-20240510.053205 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} email: null first_in_suite: false job_id: '7702358' kernel: client: sha1: distro flavor: default kdb: true sha1: fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240510.053205 ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: dry_run: true install: ceph: flavor: default sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240510.053205 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e owner: scheduled_rishabh@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4085 sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e sleep_before_teardown: 0 subset: 100/512 suite: fs suite_branch: wip-rishabh-testing-20240510.053205 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIqC8HYk0CKijYMM02BznUAsPLP2G46qUVbo5FE8y1376MkKWIo83wLGTydOd/7dxceF8ktMCwmJk5zqGgqA8A0= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKPdJxBLM8ftM75BR7CUB7fdvE3ZhDVAmg7+OAuf+oqFVgdkzvrAkAnbbBP6CebBJAMZFtoMNSn6NqWeSlGFhc= smithi064.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBw1/ycBDdWBRgaGIKpw3+ZcvU8YnDMK4AWOVD51kfirmEIkLsIH3RvFsU1ANXjfnSaLIkMBP+w2JELS+H68mJU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-11_05:57:22 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-11T12:01:38.534 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa; will attempt to use it 2024-05-11T12:01:38.535 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa/tasks 2024-05-11T12:01:38.535 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-11T12:01:38.537 INFO:teuthology.task.internal:Checking packages... 2024-05-11T12:01:38.561 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '308506802b0bdea4df7050cfc1fcbdc04fea6a6e' 2024-05-11T12:01:38.561 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-11T12:01:38.561 INFO:teuthology.packaging:ref: None 2024-05-11T12:01:38.561 INFO:teuthology.packaging:tag: None 2024-05-11T12:01:38.561 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240510.053205 2024-05-11T12:01:38.561 INFO:teuthology.packaging:sha1: 308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:01:38.561 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240510.053205 2024-05-11T12:01:38.700 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3601.g30850680 2024-05-11T12:01:38.702 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-11T12:01:38.709 INFO:teuthology.task.internal:no buildpackages task found 2024-05-11T12:01:38.709 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-11T12:01:38.734 INFO:teuthology.task.internal:Saving configuration 2024-05-11T12:01:38.753 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-11T12:01:38.759 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-11T12:01:38.786 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 11:56:45.234804', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIqC8HYk0CKijYMM02BznUAsPLP2G46qUVbo5FE8y1376MkKWIo83wLGTydOd/7dxceF8ktMCwmJk5zqGgqA8A0='} 2024-05-11T12:01:38.811 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 11:56:45.237752', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIKPdJxBLM8ftM75BR7CUB7fdvE3ZhDVAmg7+OAuf+oqFVgdkzvrAkAnbbBP6CebBJAMZFtoMNSn6NqWeSlGFhc='} 2024-05-11T12:01:38.835 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi064.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-11 11:56:45.236623', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBw1/ycBDdWBRgaGIKpw3+ZcvU8YnDMK4AWOVD51kfirmEIkLsIH3RvFsU1ANXjfnSaLIkMBP+w2JELS+H68mJU='} 2024-05-11T12:01:38.836 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-11T12:01:38.842 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-11T12:01:38.842 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-11T12:01:38.842 INFO:teuthology.task.internal:roles: ubuntu@smithi064.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-11T12:01:38.842 INFO:teuthology.run_tasks:Running task console_log... 2024-05-11T12:01:39.096 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f47f1f17160>, signals=[15]) 2024-05-11T12:01:39.096 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-11T12:01:39.277 INFO:teuthology.task.internal:Opening connections... 2024-05-11T12:01:39.278 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-05-11T12:01:39.279 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:01:39.363 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-05-11T12:01:39.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:01:39.438 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi064.front.sepia.ceph.com 2024-05-11T12:01:39.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:01:39.515 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-11T12:01:39.522 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-05-11T12:01:39.540 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-05-11T12:01:39.541 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-05-11T12:01:39.599 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-05-11T12:01:39.599 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-05-11T12:01:39.599 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-05-11T12:01:39.599 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-05-11T12:01:39.599 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-05-11T12:01:39.599 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T12:01:39.600 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T12:01:39.601 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-05-11T12:01:39.628 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-05-11T12:01:39.647 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-05-11T12:01:39.647 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T12:01:39.705 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-05-11T12:01:39.706 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-05-11T12:01:39.706 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T12:01:39.706 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-05-11T12:01:39.706 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T12:01:39.706 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T12:01:39.706 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T12:01:39.706 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-05-11T12:01:39.736 DEBUG:teuthology.orchestra.run.smithi064:> uname -m 2024-05-11T12:01:39.755 INFO:teuthology.orchestra.run.smithi064.stdout:x86_64 2024-05-11T12:01:39.755 DEBUG:teuthology.orchestra.run.smithi064:> cat /etc/os-release 2024-05-11T12:01:39.813 INFO:teuthology.orchestra.run.smithi064.stdout:NAME="CentOS Stream" 2024-05-11T12:01:39.813 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION="9" 2024-05-11T12:01:39.813 INFO:teuthology.orchestra.run.smithi064.stdout:ID="centos" 2024-05-11T12:01:39.813 INFO:teuthology.orchestra.run.smithi064.stdout:ID_LIKE="rhel fedora" 2024-05-11T12:01:39.814 INFO:teuthology.orchestra.run.smithi064.stdout:VERSION_ID="9" 2024-05-11T12:01:39.814 INFO:teuthology.orchestra.run.smithi064.stdout:PLATFORM_ID="platform:el9" 2024-05-11T12:01:39.814 INFO:teuthology.orchestra.run.smithi064.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-11T12:01:39.814 INFO:teuthology.orchestra.run.smithi064.stdout:ANSI_COLOR="0;31" 2024-05-11T12:01:39.814 INFO:teuthology.orchestra.run.smithi064.stdout:LOGO="fedora-logo-icon" 2024-05-11T12:01:39.814 INFO:teuthology.orchestra.run.smithi064.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-11T12:01:39.815 INFO:teuthology.orchestra.run.smithi064.stdout:HOME_URL="https://centos.org/" 2024-05-11T12:01:39.815 INFO:teuthology.orchestra.run.smithi064.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-11T12:01:39.815 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-11T12:01:39.815 INFO:teuthology.orchestra.run.smithi064.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-11T12:01:39.815 INFO:teuthology.lock.ops:Updating smithi064.front.sepia.ceph.com on lock server 2024-05-11T12:01:39.844 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-11T12:01:39.854 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-11T12:01:39.860 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-11T12:01:39.860 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-05-11T12:01:39.863 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-05-11T12:01:39.865 DEBUG:teuthology.orchestra.run.smithi064:> test '!' -e /home/ubuntu/cephtest 2024-05-11T12:01:39.882 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-11T12:01:39.888 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-11T12:01:39.888 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-05-11T12:01:39.921 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-05-11T12:01:39.923 DEBUG:teuthology.orchestra.run.smithi064:> test -z $(ls -A /var/lib/ceph) 2024-05-11T12:01:39.948 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-11T12:01:40.020 INFO:teuthology.run_tasks:Running task kernel... 2024-05-11T12:01:40.035 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'fb9a6a44107e0b9f9ca2d9a78f2308f5dd49911e'} 2024-05-11T12:01:40.036 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-11T12:01:40.036 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-11T12:01:40.036 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T12:01:40.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:01:40.054 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-05-11T12:01:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-437.el9.x86_64 2024-05-11T12:01:40.112 INFO:teuthology.task.kernel:Running kernel on smithi047: 5.14.0-437.el9.x86_64 2024-05-11T12:01:40.112 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum install -y kernel 2024-05-11T12:01:43.607 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-11T12:01:47.066 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-05-11T12:01:52.442 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.2 MB 00:01 2024-05-11T12:01:53.892 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-05-11T12:01:55.288 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-05-11T12:02:00.586 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-05-11T12:02:02.666 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-11T12:02:02.667 INFO:teuthology.orchestra.run.smithi047.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-11T12:02:02.766 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: Package Architecture Version Repository Size 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-11T12:02:02.783 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout:================================================================================ 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout:Install 4 Packages 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 78 M 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout:Installed size: 123 M 2024-05-11T12:02:02.784 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-05-11T12:02:03.128 INFO:teuthology.orchestra.run.smithi047.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.5 MB/s | 484 kB 00:00 2024-05-11T12:02:03.687 INFO:teuthology.orchestra.run.smithi047.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 18 MB/s | 15 MB 00:00 2024-05-11T12:02:04.337 INFO:teuthology.orchestra.run.smithi047.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 22 MB/s | 34 MB 00:01 2024-05-11T12:02:04.679 INFO:teuthology.orchestra.run.smithi047.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 18 MB/s | 28 MB 00:01 2024-05-11T12:02:04.679 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-05-11T12:02:04.680 INFO:teuthology.orchestra.run.smithi047.stdout:Total 41 MB/s | 78 MB 00:01 2024-05-11T12:02:05.391 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-05-11T12:02:05.777 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-05-11T12:02:05.777 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-05-11T12:02:06.731 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-05-11T12:02:06.732 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-05-11T12:02:09.271 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-05-11T12:02:09.912 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-11T12:02:09.996 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T12:02:12.819 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T12:02:12.956 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T12:02:18.045 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T12:02:18.093 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T12:02:23.177 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T12:02:54.094 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T12:02:54.135 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-11T12:02:54.435 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-11T12:02:54.435 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-11T12:02:54.435 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-11T12:02:54.435 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:02:55.207 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-05-11T12:02:55.541 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T12:02:55.640 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T12:02:55.640 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T12:02:55.640 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-11T12:02:55.640 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-11T12:02:55.640 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-11T12:02:55.640 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-11T12:02:55.641 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=None, version=distro) 2024-05-11T12:02:55.641 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q kernel | sort -rV | head -n 1 2024-05-11T12:02:55.691 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-11T12:02:55.692 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-11T12:02:55.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 68.el9 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8264254 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-11T12:02:55.723 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:02:55.724 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-05-11T12:02:55.725 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-11T12:02:55.725 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 5.14.0-444.el9.x86_64 2024-05-11T12:02:55.725 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-11T12:02:56.343 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-05-11T12:02:57.830 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-11T12:02:57.855 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-05-11T12:02:57.857 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-05-11T12:02:57.872 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.RVjupa3OZd 2024-05-11T12:02:57.872 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RVjupa3OZd 2024-05-11T12:02:57.939 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.RVjupa3OZd 2024-05-11T12:02:58.076 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.RVjupa3OZd is 6KB 2024-05-11T12:02:58.125 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.RVjupa3OZd 2024-05-11T12:02:58.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-05-11T12:02:58.201 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-11T12:02:58.201 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-11T12:02:58.201 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-11T12:02:58.202 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-11T12:02:58.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-11T12:02:58.354 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-05-11T12:03:28.357 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-11T12:03:28.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-05-11T12:03:28.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:03:46.846 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-05-11T12:03:55.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-05-11T12:03:55.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:04:55.851 DEBUG:teuthology.orchestra.remote:timed out 2024-05-11T12:05:07.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-05-11T12:05:07.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:05:08.122 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-05-11T12:05:08.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-05-11T12:05:08.489 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-11T12:05:08.490 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-05-11T12:05:08.507 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-444.el9.x86_64 2024-05-11T12:05:08.507 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-11T12:05:08.507 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-11T12:05:08.507 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-11T12:05:09.508 DEBUG:teuthology.parallel:result is None 2024-05-11T12:05:09.509 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-11T12:05:09.516 INFO:teuthology.task.internal:Creating test directory... 2024-05-11T12:05:09.517 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T12:05:09.520 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T12:05:09.522 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-11T12:05:09.540 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-11T12:05:09.547 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-11T12:05:09.557 INFO:teuthology.task.internal:Creating archive directory... 2024-05-11T12:05:09.558 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T12:05:09.578 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T12:05:09.581 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-11T12:05:09.613 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-11T12:05:09.620 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-11T12:05:09.620 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T12:05:09.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:05:09.663 DEBUG:teuthology.orchestra.run.smithi064:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-11T12:05:09.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:05:09.680 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T12:05:09.683 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T12:05:09.706 DEBUG:teuthology.orchestra.run.smithi064:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-11T12:05:09.741 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:05:09.745 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:05:09.755 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:05:09.773 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:05:09.786 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:05:09.788 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-11T12:05:09.790 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-11T12:05:09.798 INFO:teuthology.task.internal:Configuring sudo... 2024-05-11T12:05:09.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T12:05:09.800 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T12:05:09.817 DEBUG:teuthology.orchestra.run.smithi064:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-11T12:05:09.855 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-11T12:05:09.864 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-11T12:05:09.865 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T12:05:09.870 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T12:05:09.885 DEBUG:teuthology.orchestra.run.smithi064:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-11T12:05:09.912 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:05:09.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:05:10.023 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:05:10.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:05:10.152 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:05:10.152 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T12:05:10.219 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:05:10.264 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:05:10.313 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:05:10.396 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:05:10.442 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:05:10.442 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T12:05:10.511 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:05:10.557 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-11T12:05:10.606 DEBUG:teuthology.orchestra.run.smithi064:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:05:10.691 DEBUG:teuthology.orchestra.run.smithi064:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-11T12:05:10.732 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:05:10.732 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-11T12:05:10.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-05-11T12:05:10.802 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-05-11T12:05:10.804 DEBUG:teuthology.orchestra.run.smithi064:> sudo service rsyslog restart 2024-05-11T12:05:10.846 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T12:05:10.847 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T12:05:10.883 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T12:05:11.406 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-11T12:05:11.414 INFO:teuthology.task.internal:Starting timer... 2024-05-11T12:05:11.415 INFO:teuthology.run_tasks:Running task pcp... 2024-05-11T12:05:11.431 INFO:teuthology.run_tasks:Running task selinux... 2024-05-11T12:05:11.438 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-11T12:05:11.439 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-05-11T12:05:11.514 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-05-11T12:05:11.516 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-05-11T12:05:11.589 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-05-11T12:05:11.591 DEBUG:teuthology.orchestra.run.smithi064:> sudo service auditd rotate 2024-05-11T12:05:11.659 INFO:teuthology.orchestra.run.smithi064.stdout:Rotating logs: 2024-05-11T12:05:11.661 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-11T12:05:11.661 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-05-11T12:05:11.691 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-05-11T12:05:11.691 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-05-11T12:05:11.714 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-05-11T12:05:11.714 DEBUG:teuthology.orchestra.run.smithi064:> /usr/sbin/getenforce 2024-05-11T12:05:11.747 INFO:teuthology.orchestra.run.smithi064.stdout:Permissive 2024-05-11T12:05:11.747 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi050.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi064.front.sepia.ceph.com': 'permissive'} 2024-05-11T12:05:11.747 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T12:05:11.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:05:11.779 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T12:05:11.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:05:11.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T12:05:11.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:05:11.834 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-11T12:05:11.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-05-11T12:05:11.868 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-05-11T12:05:11.893 DEBUG:teuthology.orchestra.run.smithi064:> sudo /usr/sbin/setenforce permissive 2024-05-11T12:05:11.928 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-11T12:05:11.937 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-11T12:05:11.943 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-11T12:05:12.067 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-11T12:05:12.082 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-11T12:05:12.084 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi050.front.sepia.ceph.com,smithi064.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-11T12:09:36.951 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com'), Remote(name='ubuntu@smithi064.front.sepia.ceph.com')] 2024-05-11T12:09:36.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-05-11T12:09:36.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:09:37.030 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-05-11T12:09:37.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-05-11T12:09:37.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-11T12:09:37.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:09:37.186 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-05-11T12:09:37.225 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-11T12:09:37.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-11T12:09:37.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi064.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-11T12:09:37.301 DEBUG:teuthology.orchestra.run.smithi064:> true 2024-05-11T12:09:37.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi064.front.sepia.ceph.com' 2024-05-11T12:09:37.388 INFO:teuthology.run_tasks:Running task clock... 2024-05-11T12:09:37.398 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-11T12:09:37.399 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T12:09:37.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T12:09:37.402 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T12:09:37.402 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T12:09:37.404 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-11T12:09:37.404 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T12:09:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T12:09:37.433 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T12:09:37.447 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T12:09:37.447 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T12:09:37.460 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-11T12:09:37.471 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-05-11T12:09:37.473 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-05-11T12:09:37.475 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-11T12:09:37.482 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-05-11T12:09:37.485 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-05-11T12:09:37.497 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T12:09:37.500 INFO:teuthology.orchestra.run.smithi064.stderr:sudo: ntpd: command not found 2024-05-11T12:09:37.500 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T12:09:37.510 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T12:09:37.511 INFO:teuthology.orchestra.run.smithi064.stdout:506 Cannot talk to daemon 2024-05-11T12:09:37.515 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T12:09:37.526 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-11T12:09:37.540 INFO:teuthology.orchestra.run.smithi064.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-11T12:09:37.572 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-11T12:09:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T12:09:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-11T12:09:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.576 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.580 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-05-11T12:09:37.584 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T12:09:37.584 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-05-11T12:09:37.584 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.584 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.584 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.585 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.603 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-05-11T12:09:37.606 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T12:09:37.606 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-11T12:09:37.606 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.607 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.607 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.607 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T12:09:37.607 INFO:teuthology.run_tasks:Running task install... 2024-05-11T12:09:37.620 DEBUG:teuthology.task.install:project ceph 2024-05-11T12:09:37.620 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'}} 2024-05-11T12:09:37.620 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'} 2024-05-11T12:09:37.620 INFO:teuthology.task.install:Using flavor: default 2024-05-11T12:09:37.627 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-11T12:09:37.628 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-11T12:09:37.628 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T12:09:37.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:09:37.630 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T12:09:37.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:09:37.631 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'tag': None, 'wait_for_package': False} 2024-05-11T12:09:37.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:09:37.853 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T12:09:37.853 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T12:09:37.857 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T12:09:37.857 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T12:09:37.860 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/ 2024-05-11T12:09:37.860 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3601.g30850680 2024-05-11T12:09:37.971 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T12:09:37.972 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:09:37.972 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T12:09:37.998 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T12:09:37.998 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T12:09:38.023 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T12:09:38.023 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:09:38.023 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T12:09:38.050 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T12:09:38.050 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T12:09:38.063 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T12:09:38.118 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T12:09:38.138 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T12:09:38.191 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-11T12:09:38.192 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:09:38.192 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-11T12:09:38.194 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T12:09:38.201 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-05-11T12:09:38.203 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-05-11T12:09:38.222 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-11T12:09:38.222 DEBUG:teuthology.orchestra.run.smithi064:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-11T12:09:38.260 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-05-11T12:09:38.262 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-05-11T12:09:38.291 DEBUG:teuthology.orchestra.run.smithi064:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-11T12:09:38.369 DEBUG:teuthology.orchestra.run.smithi064:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-11T12:09:38.434 INFO:teuthology.orchestra.run.smithi064.stdout:check_obsoletes = 1 2024-05-11T12:09:38.436 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum clean all 2024-05-11T12:09:38.439 INFO:teuthology.orchestra.run.smithi050.stdout:54 files removed 2024-05-11T12:09:38.466 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T12:09:38.497 INFO:teuthology.orchestra.run.smithi047.stdout:54 files removed 2024-05-11T12:09:38.526 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T12:09:38.675 INFO:teuthology.orchestra.run.smithi064.stdout:54 files removed 2024-05-11T12:09:38.703 DEBUG:teuthology.orchestra.run.smithi064:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-11T12:09:39.299 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 150 kB/s | 84 kB 00:00 2024-05-11T12:09:39.316 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 167 kB/s | 84 kB 00:00 2024-05-11T12:09:39.342 INFO:teuthology.orchestra.run.smithi064.stdout:ceph packages for x86_64 230 kB/s | 84 kB 00:00 2024-05-11T12:09:39.667 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-05-11T12:09:39.733 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-05-11T12:09:39.776 INFO:teuthology.orchestra.run.smithi064.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-11T12:09:39.975 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-05-11T12:09:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-11T12:09:40.219 INFO:teuthology.orchestra.run.smithi064.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-05-11T12:09:40.794 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-11T12:09:41.834 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.1 MB 00:01 2024-05-11T12:09:41.917 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-11T12:09:43.819 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-05-11T12:09:45.037 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 19 MB 00:03 2024-05-11T12:09:48.061 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-11T12:09:49.254 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-11T12:09:49.562 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-11T12:09:50.696 INFO:teuthology.orchestra.run.smithi064.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 16 kB 00:00 2024-05-11T12:09:51.149 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-11T12:09:52.715 INFO:teuthology.orchestra.run.smithi064.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-11T12:09:56.489 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-11T12:09:57.534 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-05-11T12:09:58.148 INFO:teuthology.orchestra.run.smithi064.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-11T12:09:58.892 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:09:58.892 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:09:58.904 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T12:09:58.905 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T12:09:58.906 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T12:09:58.907 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T12:09:58.908 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T12:09:58.911 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T12:09:58.912 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T12:09:58.978 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T12:09:58.989 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T12:09:58.990 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T12:09:58.991 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T12:09:58.992 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T12:09:58.993 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T12:09:58.994 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T12:09:58.995 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T12:09:58.996 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T12:09:58.997 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T12:09:58.998 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T12:09:58.999 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-05-11T12:09:59.000 INFO:teuthology.orchestra.run.smithi047.stdout:Install 148 Packages 2024-05-11T12:09:59.001 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 8 Packages 2024-05-11T12:09:59.001 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:09:59.001 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 226 M 2024-05-11T12:09:59.001 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-05-11T12:09:59.865 INFO:teuthology.orchestra.run.smithi047.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-05-11T12:10:00.274 INFO:teuthology.orchestra.run.smithi047.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 2.1 MB/s | 890 kB 00:00 2024-05-11T12:10:00.382 INFO:teuthology.orchestra.run.smithi047.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 1.3 MB/s | 145 kB 00:00 2024-05-11T12:10:00.617 INFO:teuthology.orchestra.run.smithi064.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:10:00.618 INFO:teuthology.orchestra.run.smithi064.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:10:00.630 INFO:teuthology.orchestra.run.smithi064.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T12:10:00.631 INFO:teuthology.orchestra.run.smithi064.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T12:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T12:10:00.632 INFO:teuthology.orchestra.run.smithi064.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T12:10:00.634 INFO:teuthology.orchestra.run.smithi064.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T12:10:00.636 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T12:10:00.637 INFO:teuthology.orchestra.run.smithi064.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T12:10:00.704 INFO:teuthology.orchestra.run.smithi064.stdout:Dependencies resolved. 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: Package Arch Version Repository Size 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout:Installing: 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T12:10:00.719 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T12:10:00.720 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T12:10:00.721 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrading: 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T12:10:00.722 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout:Installing dependencies: 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T12:10:00.723 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T12:10:00.724 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T12:10:00.725 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T12:10:00.726 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T12:10:00.727 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T12:10:00.728 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T12:10:00.729 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T12:10:00.730 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T12:10:00.731 INFO:teuthology.orchestra.run.smithi064.stdout:Installing weak dependencies: 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction Summary 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout:======================================================================================= 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout:Install 148 Packages 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout:Upgrade 8 Packages 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:10:00.732 INFO:teuthology.orchestra.run.smithi064.stdout:Total download size: 226 M 2024-05-11T12:10:00.733 INFO:teuthology.orchestra.run.smithi064.stdout:Downloading Packages: 2024-05-11T12:10:00.974 INFO:teuthology.orchestra.run.smithi047.stdout:(4/156): ceph-common-19.0.0-3601.g30850680.el9. 16 MB/s | 21 MB 00:01 2024-05-11T12:10:01.091 INFO:teuthology.orchestra.run.smithi047.stdout:(5/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 13 MB/s | 1.5 MB 00:00 2024-05-11T12:10:01.133 INFO:teuthology.orchestra.run.smithi047.stdout:(6/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 3.2 MB/s | 2.4 MB 00:00 2024-05-11T12:10:01.400 INFO:teuthology.orchestra.run.smithi047.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 15 MB/s | 4.7 MB 00:00 2024-05-11T12:10:01.733 INFO:teuthology.orchestra.run.smithi064.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-11T12:10:01.826 INFO:teuthology.orchestra.run.smithi047.stdout:(8/156): ceph-base-19.0.0-3601.g30850680.el9.x8 2.5 MB/s | 5.4 MB 00:02 2024-05-11T12:10:01.926 INFO:teuthology.orchestra.run.smithi047.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 252 kB/s | 25 kB 00:00 2024-05-11T12:10:02.059 INFO:teuthology.orchestra.run.smithi047.stdout:(10/156): ceph-radosgw-19.0.0-3601.g30850680.el 16 MB/s | 11 MB 00:00 2024-05-11T12:10:02.118 INFO:teuthology.orchestra.run.smithi047.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 3.8 MB/s | 225 kB 00:00 2024-05-11T12:10:02.185 INFO:teuthology.orchestra.run.smithi047.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 406 kB/s | 27 kB 00:00 2024-05-11T12:10:02.252 INFO:teuthology.orchestra.run.smithi047.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 477 kB/s | 32 kB 00:00 2024-05-11T12:10:02.360 INFO:teuthology.orchestra.run.smithi047.stdout:(14/156): libcephfs2-19.0.0-3601.g30850680.el9. 6.7 MB/s | 746 kB 00:00 2024-05-11T12:10:02.444 INFO:teuthology.orchestra.run.smithi047.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 1.9 MB/s | 163 kB 00:00 2024-05-11T12:10:02.516 INFO:teuthology.orchestra.run.smithi064.stdout:(2/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 1.1 MB/s | 890 kB 00:00 2024-05-11T12:10:02.536 INFO:teuthology.orchestra.run.smithi047.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 1.4 MB/s | 127 kB 00:00 2024-05-11T12:10:02.636 INFO:teuthology.orchestra.run.smithi047.stdout:(17/156): libradosstriper1-19.0.0-3601.g3085068 4.9 MB/s | 501 kB 00:00 2024-05-11T12:10:02.683 INFO:teuthology.orchestra.run.smithi064.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 868 kB/s | 145 kB 00:00 2024-05-11T12:10:02.901 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-05-11T12:10:03.287 INFO:teuthology.orchestra.run.smithi047.stdout:(18/156): ceph-osd-19.0.0-3601.g30850680.el9.x8 7.8 MB/s | 17 MB 00:02 2024-05-11T12:10:03.370 INFO:teuthology.orchestra.run.smithi047.stdout:(19/156): librgw2-19.0.0-3601.g30850680.el9.x86 7.3 MB/s | 5.4 MB 00:00 2024-05-11T12:10:03.395 INFO:teuthology.orchestra.run.smithi047.stdout:(20/156): python3-ceph-argparse-19.0.0-3601.g30 417 kB/s | 45 kB 00:00 2024-05-11T12:10:03.421 INFO:teuthology.orchestra.run.smithi047.stdout:(21/156): python3-ceph-common-19.0.0-3601.g3085 2.6 MB/s | 135 kB 00:00 2024-05-11T12:10:03.463 INFO:teuthology.orchestra.run.smithi047.stdout:(22/156): python3-cephfs-19.0.0-3601.g30850680. 2.3 MB/s | 161 kB 00:00 2024-05-11T12:10:03.488 INFO:teuthology.orchestra.run.smithi047.stdout:(23/156): python3-rados-19.0.0-3601.g30850680.e 4.7 MB/s | 321 kB 00:00 2024-05-11T12:10:03.530 INFO:teuthology.orchestra.run.smithi047.stdout:(24/156): python3-rbd-19.0.0-3601.g30850680.el9 4.4 MB/s | 300 kB 00:00 2024-05-11T12:10:03.564 INFO:teuthology.orchestra.run.smithi047.stdout:(25/156): python3-rgw-19.0.0-3601.g30850680.el9 1.3 MB/s | 100 kB 00:00 2024-05-11T12:10:03.589 INFO:teuthology.orchestra.run.smithi047.stdout:(26/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-11T12:10:03.658 INFO:teuthology.orchestra.run.smithi064.stdout:(4/156): ceph-base-19.0.0-3601.g30850680.el9.x8 2.4 MB/s | 5.4 MB 00:02 2024-05-11T12:10:03.673 INFO:teuthology.orchestra.run.smithi047.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 2.0 MB/s | 171 kB 00:00 2024-05-11T12:10:03.765 INFO:teuthology.orchestra.run.smithi047.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 374 kB/s | 34 kB 00:00 2024-05-11T12:10:03.809 INFO:teuthology.orchestra.run.smithi064.stdout:(5/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-05-11T12:10:03.865 INFO:teuthology.orchestra.run.smithi047.stdout:(29/156): rbd-mirror-19.0.0-3601.g30850680.el9. 10 MB/s | 3.1 MB 00:00 2024-05-11T12:10:03.890 INFO:teuthology.orchestra.run.smithi047.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 1.1 MB/s | 142 kB 00:00 2024-05-11T12:10:04.176 INFO:teuthology.orchestra.run.smithi064.stdout:(6/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-05-11T12:10:04.257 INFO:teuthology.orchestra.run.smithi047.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3601.g30850 9.0 MB/s | 3.5 MB 00:00 2024-05-11T12:10:04.324 INFO:teuthology.orchestra.run.smithi047.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3601.g30 4.0 MB/s | 274 kB 00:00 2024-05-11T12:10:04.383 INFO:teuthology.orchestra.run.smithi047.stdout:(33/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 846 kB/s | 49 kB 00:00 2024-05-11T12:10:04.441 INFO:teuthology.orchestra.run.smithi047.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3601.g3 282 kB/s | 16 kB 00:00 2024-05-11T12:10:04.517 INFO:teuthology.orchestra.run.smithi047.stdout:(35/156): ceph-volume-19.0.0-3601.g30850680.el9 3.6 MB/s | 273 kB 00:00 2024-05-11T12:10:04.643 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-11T12:10:04.650 INFO:teuthology.orchestra.run.smithi047.stdout:(36/156): cephadm-19.0.0-3601.g30850680.el9.noa 5.6 MB/s | 764 kB 00:00 2024-05-11T12:10:04.700 INFO:teuthology.orchestra.run.smithi047.stdout:(37/156): cephfs-top-19.0.0-3601.g30850680.el9. 365 kB/s | 18 kB 00:00 2024-05-11T12:10:04.767 INFO:teuthology.orchestra.run.smithi047.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-05-11T12:10:04.859 INFO:teuthology.orchestra.run.smithi047.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-05-11T12:10:04.943 INFO:teuthology.orchestra.run.smithi047.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-05-11T12:10:04.977 INFO:teuthology.orchestra.run.smithi047.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-05-11T12:10:05.043 INFO:teuthology.orchestra.run.smithi047.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-05-11T12:10:05.119 INFO:teuthology.orchestra.run.smithi047.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-11T12:10:05.161 INFO:teuthology.orchestra.run.smithi047.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-05-11T12:10:05.220 INFO:teuthology.orchestra.run.smithi047.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-11T12:10:05.245 INFO:teuthology.orchestra.run.smithi047.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-11T12:10:05.311 INFO:teuthology.orchestra.run.smithi047.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-05-11T12:10:05.501 INFO:teuthology.orchestra.run.smithi064.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-05-11T12:10:06.170 INFO:teuthology.orchestra.run.smithi047.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 745 kB/s | 689 kB 00:00 2024-05-11T12:10:06.201 INFO:teuthology.orchestra.run.smithi064.stdout:(8/156): ceph-common-19.0.0-3601.g30850680.el9. 4.4 MB/s | 21 MB 00:04 2024-05-11T12:10:06.288 INFO:teuthology.orchestra.run.smithi047.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 967 kB/s | 943 kB 00:00 2024-05-11T12:10:06.485 INFO:teuthology.orchestra.run.smithi064.stdout:(9/156): ceph-selinux-19.0.0-3601.g30850680.el9 89 kB/s | 25 kB 00:00 2024-05-11T12:10:06.740 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-11T12:10:06.863 INFO:teuthology.orchestra.run.smithi047.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 37 kB/s | 21 kB 00:00 2024-05-11T12:10:06.906 INFO:teuthology.orchestra.run.smithi047.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 146 kB/s | 107 kB 00:00 2024-05-11T12:10:07.452 INFO:teuthology.orchestra.run.smithi064.stdout:(10/156): ceph-osd-19.0.0-3601.g30850680.el9.x8 5.1 MB/s | 17 MB 00:03 2024-05-11T12:10:07.505 INFO:teuthology.orchestra.run.smithi047.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 54 kB/s | 33 kB 00:00 2024-05-11T12:10:07.519 INFO:teuthology.orchestra.run.smithi064.stdout:(11/156): cephfs-mirror-19.0.0-3601.g30850680.e 3.3 MB/s | 225 kB 00:00 2024-05-11T12:10:07.577 INFO:teuthology.orchestra.run.smithi064.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 464 kB/s | 27 kB 00:00 2024-05-11T12:10:07.644 INFO:teuthology.orchestra.run.smithi064.stdout:(13/156): libcephfs-devel-19.0.0-3601.g30850680 476 kB/s | 32 kB 00:00 2024-05-11T12:10:07.688 INFO:teuthology.orchestra.run.smithi047.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 380 kB/s | 313 kB 00:00 2024-05-11T12:10:07.819 INFO:teuthology.orchestra.run.smithi064.stdout:(14/156): libcephfs2-19.0.0-3601.g30850680.el9. 4.2 MB/s | 746 kB 00:00 2024-05-11T12:10:07.903 INFO:teuthology.orchestra.run.smithi064.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 1.9 MB/s | 163 kB 00:00 2024-05-11T12:10:08.245 INFO:teuthology.orchestra.run.smithi064.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 371 kB/s | 127 kB 00:00 2024-05-11T12:10:08.397 INFO:teuthology.orchestra.run.smithi047.stdout:(54/156): ceph-test-19.0.0-3601.g30850680.el9.x 7.6 MB/s | 49 MB 00:06 2024-05-11T12:10:08.420 INFO:teuthology.orchestra.run.smithi064.stdout:(17/156): ceph-radosgw-19.0.0-3601.g30850680.el 3.6 MB/s | 11 MB 00:02 2024-05-11T12:10:08.487 INFO:teuthology.orchestra.run.smithi064.stdout:(18/156): libradosstriper1-19.0.0-3601.g3085068 2.0 MB/s | 501 kB 00:00 2024-05-11T12:10:08.562 INFO:teuthology.orchestra.run.smithi064.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 603 kB/s | 45 kB 00:00 2024-05-11T12:10:08.597 INFO:teuthology.orchestra.run.smithi047.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 20 kB/s | 18 kB 00:00 2024-05-11T12:10:08.629 INFO:teuthology.orchestra.run.smithi064.stdout:(20/156): python3-ceph-common-19.0.0-3601.g3085 2.0 MB/s | 135 kB 00:00 2024-05-11T12:10:08.704 INFO:teuthology.orchestra.run.smithi064.stdout:(21/156): python3-cephfs-19.0.0-3601.g30850680. 2.1 MB/s | 161 kB 00:00 2024-05-11T12:10:08.822 INFO:teuthology.orchestra.run.smithi064.stdout:(22/156): python3-rados-19.0.0-3601.g30850680.e 2.7 MB/s | 321 kB 00:00 2024-05-11T12:10:08.914 INFO:teuthology.orchestra.run.smithi047.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-05-11T12:10:08.922 INFO:teuthology.orchestra.run.smithi064.stdout:(23/156): python3-rbd-19.0.0-3601.g30850680.el9 2.9 MB/s | 300 kB 00:00 2024-05-11T12:10:09.239 INFO:teuthology.orchestra.run.smithi064.stdout:(24/156): python3-rgw-19.0.0-3601.g30850680.el9 314 kB/s | 100 kB 00:00 2024-05-11T12:10:09.298 INFO:teuthology.orchestra.run.smithi064.stdout:(25/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.4 MB/s | 86 kB 00:00 2024-05-11T12:10:09.373 INFO:teuthology.orchestra.run.smithi047.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 207 kB/s | 160 kB 00:00 2024-05-11T12:10:09.540 INFO:teuthology.orchestra.run.smithi047.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 72 kB/s | 45 kB 00:00 2024-05-11T12:10:09.631 INFO:teuthology.orchestra.run.smithi064.stdout:(26/156): librgw2-19.0.0-3601.g30850680.el9.x86 4.4 MB/s | 5.4 MB 00:01 2024-05-11T12:10:09.690 INFO:teuthology.orchestra.run.smithi064.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 2.9 MB/s | 171 kB 00:00 2024-05-11T12:10:09.756 INFO:teuthology.orchestra.run.smithi064.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3601.g 515 kB/s | 34 kB 00:00 2024-05-11T12:10:09.824 INFO:teuthology.orchestra.run.smithi064.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 2.1 MB/s | 142 kB 00:00 2024-05-11T12:10:09.916 INFO:teuthology.orchestra.run.smithi064.stdout:(30/156): rbd-mirror-19.0.0-3601.g30850680.el9. 5.0 MB/s | 3.1 MB 00:00 2024-05-11T12:10:10.032 INFO:teuthology.orchestra.run.smithi047.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.2 MB/s | 3.0 MB 00:02 2024-05-11T12:10:10.415 INFO:teuthology.orchestra.run.smithi047.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 636 kB/s | 662 kB 00:01 2024-05-11T12:10:10.474 INFO:teuthology.orchestra.run.smithi047.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 624 kB/s | 582 kB 00:00 2024-05-11T12:10:10.557 INFO:teuthology.orchestra.run.smithi064.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3601.g30850 4.8 MB/s | 3.5 MB 00:00 2024-05-11T12:10:10.658 INFO:teuthology.orchestra.run.smithi064.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3601.g30 2.7 MB/s | 274 kB 00:00 2024-05-11T12:10:10.716 INFO:teuthology.orchestra.run.smithi064.stdout:(33/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 846 kB/s | 49 kB 00:00 2024-05-11T12:10:10.774 INFO:teuthology.orchestra.run.smithi047.stdout:(62/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 34 kB/s | 25 kB 00:00 2024-05-11T12:10:10.783 INFO:teuthology.orchestra.run.smithi064.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3601.g3 246 kB/s | 16 kB 00:00 2024-05-11T12:10:10.884 INFO:teuthology.orchestra.run.smithi064.stdout:(35/156): ceph-volume-19.0.0-3601.g30850680.el9 2.7 MB/s | 273 kB 00:00 2024-05-11T12:10:11.091 INFO:teuthology.orchestra.run.smithi047.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 70 kB/s | 47 kB 00:00 2024-05-11T12:10:11.126 INFO:teuthology.orchestra.run.smithi064.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 6.1 MB/s | 7.4 MB 00:01 2024-05-11T12:10:11.159 INFO:teuthology.orchestra.run.smithi064.stdout:(37/156): cephadm-19.0.0-3601.g30850680.el9.noa 2.7 MB/s | 764 kB 00:00 2024-05-11T12:10:11.268 INFO:teuthology.orchestra.run.smithi064.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 741 kB/s | 80 kB 00:00 2024-05-11T12:10:11.408 INFO:teuthology.orchestra.run.smithi047.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 313 kB/s | 292 kB 00:00 2024-05-11T12:10:11.575 INFO:teuthology.orchestra.run.smithi047.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 244 kB/s | 188 kB 00:00 2024-05-11T12:10:11.700 INFO:teuthology.orchestra.run.smithi047.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 58 kB/s | 35 kB 00:00 2024-05-11T12:10:11.702 INFO:teuthology.orchestra.run.smithi064.stdout:(39/156): ceph-test-19.0.0-3601.g30850680.el9.x 9.4 MB/s | 49 MB 00:05 2024-05-11T12:10:11.719 INFO:teuthology.orchestra.run.smithi064.stdout:(40/156): cephfs-top-19.0.0-3601.g30850680.el9. 31 kB/s | 18 kB 00:00 2024-05-11T12:10:11.769 INFO:teuthology.orchestra.run.smithi064.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-05-11T12:10:11.852 INFO:teuthology.orchestra.run.smithi064.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-05-11T12:10:11.878 INFO:teuthology.orchestra.run.smithi064.stdout:(43/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-05-11T12:10:11.903 INFO:teuthology.orchestra.run.smithi064.stdout:(44/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-11T12:10:11.928 INFO:teuthology.orchestra.run.smithi064.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-05-11T12:10:11.954 INFO:teuthology.orchestra.run.smithi064.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-11T12:10:11.979 INFO:teuthology.orchestra.run.smithi064.stdout:(47/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-11T12:10:12.004 INFO:teuthology.orchestra.run.smithi064.stdout:(48/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-11T12:10:12.229 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-05-11T12:10:12.609 INFO:teuthology.orchestra.run.smithi047.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.9 MB/s | 4.7 MB 00:01 2024-05-11T12:10:12.750 INFO:teuthology.orchestra.run.smithi047.stdout:(68/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-05-11T12:10:12.754 INFO:teuthology.orchestra.run.smithi064.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 142 kB/s | 107 kB 00:00 2024-05-11T12:10:12.784 INFO:teuthology.orchestra.run.smithi047.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-05-11T12:10:12.896 INFO:teuthology.orchestra.run.smithi064.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 731 kB/s | 689 kB 00:00 2024-05-11T12:10:13.046 INFO:teuthology.orchestra.run.smithi064.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 883 kB/s | 943 kB 00:01 2024-05-11T12:10:13.355 INFO:teuthology.orchestra.run.smithi064.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 36 kB/s | 21 kB 00:00 2024-05-11T12:10:13.518 INFO:teuthology.orchestra.run.smithi047.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 174 kB/s | 133 kB 00:00 2024-05-11T12:10:13.584 INFO:teuthology.orchestra.run.smithi047.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 934 kB/s | 911 kB 00:00 2024-05-11T12:10:13.680 INFO:teuthology.orchestra.run.smithi064.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 52 kB/s | 33 kB 00:00 2024-05-11T12:10:13.730 INFO:teuthology.orchestra.run.smithi064.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 376 kB/s | 313 kB 00:00 2024-05-11T12:10:14.060 INFO:teuthology.orchestra.run.smithi047.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-05-11T12:10:14.263 INFO:teuthology.orchestra.run.smithi064.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 31 kB/s | 18 kB 00:00 2024-05-11T12:10:14.277 INFO:teuthology.orchestra.run.smithi047.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 126 kB/s | 87 kB 00:00 2024-05-11T12:10:14.306 INFO:teuthology.orchestra.run.smithi064.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 19 kB/s | 11 kB 00:00 2024-05-11T12:10:14.527 INFO:teuthology.orchestra.run.smithi047.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-05-11T12:10:14.614 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:10:14.615 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-11T12:10:14.626 INFO:teuthology.orchestra.run.smithi050.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-11T12:10:14.627 INFO:teuthology.orchestra.run.smithi050.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-11T12:10:14.628 INFO:teuthology.orchestra.run.smithi050.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-11T12:10:14.629 INFO:teuthology.orchestra.run.smithi050.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-11T12:10:14.630 INFO:teuthology.orchestra.run.smithi050.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-11T12:10:14.633 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-11T12:10:14.634 INFO:teuthology.orchestra.run.smithi050.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-11T12:10:14.699 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-11T12:10:14.706 INFO:teuthology.orchestra.run.smithi064.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-05-11T12:10:14.715 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-11T12:10:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-11T12:10:14.715 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-11T12:10:14.715 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-11T12:10:14.715 INFO:teuthology.orchestra.run.smithi050.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-3601.g30850680.el9 ceph 6.6 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 890 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3601.g30850680.el9 ceph 145 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-3601.g30850680.el9 ceph 1.5 M 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 142 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 3.5 M 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 7.4 M 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 49 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 11 M 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-3601.g30850680.el9 ceph 49 M 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 273 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 764 k 2024-05-11T12:10:14.716 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 225 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell x86_64 2:19.0.0-3601.g30850680.el9 ceph 27 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 18 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 32 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 746 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-3601.g30850680.el9 ceph 127 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-11T12:10:14.717 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-3601.g30850680.el9 ceph 161 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-3601.g30850680.el9 ceph 321 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 300 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-3601.g30850680.el9 ceph 100 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-3601.g30850680.el9 ceph 86 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-3601.g30850680.el9 ceph 171 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-11T12:10:14.718 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.4 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 3.1 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 21 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 34 k 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-3601.g30850680.el9 ceph 2.4 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 274 k 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-3601.g30850680.el9 ceph 4.7 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-3601.g30850680.el9 ceph 17 M 2024-05-11T12:10:14.719 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3601.g30850680.el9 ceph-noarch 16 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-3601.g30850680.el9 ceph 25 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-3601.g30850680.el9 ceph 163 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-11T12:10:14.720 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-3601.g30850680.el9 ceph 501 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-3601.g30850680.el9 ceph 5.4 M 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-11T12:10:14.721 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-3601.g30850680.el9 ceph 45 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-3601.g30850680.el9 ceph 135 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-11T12:10:14.722 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-11T12:10:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-11T12:10:14.724 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-11T12:10:14.725 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-11T12:10:14.726 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout:Install 148 Packages 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 8 Packages 2024-05-11T12:10:14.727 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:10:14.728 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 226 M 2024-05-11T12:10:14.728 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-11T12:10:15.144 INFO:teuthology.orchestra.run.smithi047.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 282 kB/s | 244 kB 00:00 2024-05-11T12:10:15.157 INFO:teuthology.orchestra.run.smithi064.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 179 kB/s | 160 kB 00:00 2024-05-11T12:10:15.182 INFO:teuthology.orchestra.run.smithi064.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 52 kB/s | 45 kB 00:00 2024-05-11T12:10:15.327 INFO:teuthology.orchestra.run.smithi047.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-05-11T12:10:15.378 INFO:teuthology.orchestra.run.smithi047.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 301 kB/s | 248 kB 00:00 2024-05-11T12:10:15.506 INFO:teuthology.orchestra.run.smithi050.stdout:(1/156): ceph-19.0.0-3601.g30850680.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-11T12:10:15.682 INFO:teuthology.orchestra.run.smithi064.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 700 kB/s | 662 kB 00:00 2024-05-11T12:10:15.762 INFO:teuthology.orchestra.run.smithi047.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 42 kB/s | 26 kB 00:00 2024-05-11T12:10:15.783 INFO:teuthology.orchestra.run.smithi064.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 42 kB/s | 25 kB 00:00 2024-05-11T12:10:15.854 INFO:teuthology.orchestra.run.smithi047.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-05-11T12:10:15.898 INFO:teuthology.orchestra.run.smithi050.stdout:(2/156): ceph-base-19.0.0-3601.g30850680.el9.x8 8.5 MB/s | 5.4 MB 00:00 2024-05-11T12:10:16.050 INFO:teuthology.orchestra.run.smithi064.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 653 kB/s | 582 kB 00:00 2024-05-11T12:10:16.146 INFO:teuthology.orchestra.run.smithi047.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 224 kB/s | 172 kB 00:00 2024-05-11T12:10:16.206 INFO:teuthology.orchestra.run.smithi050.stdout:(3/156): ceph-immutable-object-cache-19.0.0-360 470 kB/s | 145 kB 00:00 2024-05-11T12:10:16.265 INFO:teuthology.orchestra.run.smithi050.stdout:(4/156): ceph-fuse-19.0.0-3601.g30850680.el9.x8 1.1 MB/s | 890 kB 00:00 2024-05-11T12:10:16.367 INFO:teuthology.orchestra.run.smithi064.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-05-11T12:10:16.388 INFO:teuthology.orchestra.run.smithi047.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-05-11T12:10:16.549 INFO:teuthology.orchestra.run.smithi050.stdout:(5/156): ceph-mds-19.0.0-3601.g30850680.el9.x86 6.9 MB/s | 2.4 MB 00:00 2024-05-11T12:10:16.650 INFO:teuthology.orchestra.run.smithi064.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 338 kB/s | 292 kB 00:00 2024-05-11T12:10:16.757 INFO:teuthology.orchestra.run.smithi050.stdout:(6/156): ceph-common-19.0.0-3601.g30850680.el9. 14 MB/s | 21 MB 00:01 2024-05-11T12:10:16.850 INFO:teuthology.orchestra.run.smithi064.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 244 kB/s | 188 kB 00:00 2024-05-11T12:10:16.967 INFO:teuthology.orchestra.run.smithi064.stdout:(66/156): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-11T12:10:16.996 INFO:teuthology.orchestra.run.smithi047.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 295 kB/s | 250 kB 00:00 2024-05-11T12:10:17.041 INFO:teuthology.orchestra.run.smithi050.stdout:(7/156): ceph-mon-19.0.0-3601.g30850680.el9.x86 9.6 MB/s | 4.7 MB 00:00 2024-05-11T12:10:17.079 INFO:teuthology.orchestra.run.smithi047.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-05-11T12:10:17.213 INFO:teuthology.orchestra.run.smithi047.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 279 kB/s | 222 kB 00:00 2024-05-11T12:10:17.713 INFO:teuthology.orchestra.run.smithi047.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 108 kB/s | 77 kB 00:00 2024-05-11T12:10:17.739 INFO:teuthology.orchestra.run.smithi047.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 64 kB/s | 42 kB 00:00 2024-05-11T12:10:17.766 INFO:teuthology.orchestra.run.smithi050.stdout:(8/156): ceph-mgr-19.0.0-3601.g30850680.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-05-11T12:10:17.933 INFO:teuthology.orchestra.run.smithi050.stdout:(9/156): ceph-osd-19.0.0-3601.g30850680.el9.x86 14 MB/s | 17 MB 00:01 2024-05-11T12:10:17.958 INFO:teuthology.orchestra.run.smithi050.stdout:(10/156): ceph-selinux-19.0.0-3601.g30850680.el 131 kB/s | 25 kB 00:00 2024-05-11T12:10:17.994 INFO:teuthology.orchestra.run.smithi064.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.5 MB/s | 4.7 MB 00:01 2024-05-11T12:10:18.022 INFO:teuthology.orchestra.run.smithi047.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 333 kB/s | 269 kB 00:00 2024-05-11T12:10:18.125 INFO:teuthology.orchestra.run.smithi050.stdout:(11/156): ceph-radosgw-19.0.0-3601.g30850680.el 9.8 MB/s | 11 MB 00:01 2024-05-11T12:10:18.176 INFO:teuthology.orchestra.run.smithi064.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-05-11T12:10:18.184 INFO:teuthology.orchestra.run.smithi050.stdout:(12/156): cephfs-shell-19.0.0-3601.g30850680.el 465 kB/s | 27 kB 00:00 2024-05-11T12:10:18.209 INFO:teuthology.orchestra.run.smithi050.stdout:(13/156): cephfs-mirror-19.0.0-3601.g30850680.e 898 kB/s | 225 kB 00:00 2024-05-11T12:10:18.243 INFO:teuthology.orchestra.run.smithi064.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-05-11T12:10:18.260 INFO:teuthology.orchestra.run.smithi050.stdout:(14/156): libcephfs-devel-19.0.0-3601.g30850680 421 kB/s | 32 kB 00:00 2024-05-11T12:10:18.335 INFO:teuthology.orchestra.run.smithi050.stdout:(15/156): libcephsqlite-19.0.0-3601.g30850680.e 2.1 MB/s | 163 kB 00:00 2024-05-11T12:10:18.410 INFO:teuthology.orchestra.run.smithi050.stdout:(16/156): librados-devel-19.0.0-3601.g30850680. 1.7 MB/s | 127 kB 00:00 2024-05-11T12:10:18.481 INFO:teuthology.orchestra.run.smithi047.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 207 kB/s | 159 kB 00:00 2024-05-11T12:10:18.510 INFO:teuthology.orchestra.run.smithi050.stdout:(17/156): libradosstriper1-19.0.0-3601.g3085068 4.9 MB/s | 501 kB 00:00 2024-05-11T12:10:18.556 INFO:teuthology.orchestra.run.smithi047.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 341 kB/s | 279 kB 00:00 2024-05-11T12:10:18.665 INFO:teuthology.orchestra.run.smithi047.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-05-11T12:10:18.836 INFO:teuthology.orchestra.run.smithi050.stdout:(18/156): libcephfs2-19.0.0-3601.g30850680.el9. 1.2 MB/s | 746 kB 00:00 2024-05-11T12:10:18.943 INFO:teuthology.orchestra.run.smithi064.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 174 kB/s | 133 kB 00:00 2024-05-11T12:10:18.953 INFO:teuthology.orchestra.run.smithi050.stdout:(19/156): python3-ceph-argparse-19.0.0-3601.g30 387 kB/s | 45 kB 00:00 2024-05-11T12:10:19.002 INFO:teuthology.orchestra.run.smithi064.stdout:(71/156): postgresql-contrib-13.14-1.el9.x86_64 905 kB/s | 911 kB 00:01 2024-05-11T12:10:19.112 INFO:teuthology.orchestra.run.smithi050.stdout:(20/156): librgw2-19.0.0-3601.g30850680.el9.x86 9.0 MB/s | 5.4 MB 00:00 2024-05-11T12:10:19.145 INFO:teuthology.orchestra.run.smithi050.stdout:(21/156): python3-ceph-common-19.0.0-3601.g3085 706 kB/s | 135 kB 00:00 2024-05-11T12:10:19.174 INFO:teuthology.orchestra.run.smithi047.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 77 kB/s | 54 kB 00:00 2024-05-11T12:10:19.195 INFO:teuthology.orchestra.run.smithi050.stdout:(22/156): python3-cephfs-19.0.0-3601.g30850680. 1.9 MB/s | 161 kB 00:00 2024-05-11T12:10:19.279 INFO:teuthology.orchestra.run.smithi050.stdout:(23/156): python3-rbd-19.0.0-3601.g30850680.el9 3.5 MB/s | 300 kB 00:00 2024-05-11T12:10:19.290 INFO:teuthology.orchestra.run.smithi047.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 67 kB/s | 42 kB 00:00 2024-05-11T12:10:19.354 INFO:teuthology.orchestra.run.smithi050.stdout:(24/156): python3-rgw-19.0.0-3601.g30850680.el9 1.3 MB/s | 100 kB 00:00 2024-05-11T12:10:19.429 INFO:teuthology.orchestra.run.smithi050.stdout:(25/156): rbd-fuse-19.0.0-3601.g30850680.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-11T12:10:19.455 INFO:teuthology.orchestra.run.smithi050.stdout:(26/156): python3-rados-19.0.0-3601.g30850680.e 1.0 MB/s | 321 kB 00:00 2024-05-11T12:10:19.527 INFO:teuthology.orchestra.run.smithi064.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-05-11T12:10:19.664 INFO:teuthology.orchestra.run.smithi050.stdout:(27/156): rbd-nbd-19.0.0-3601.g30850680.el9.x86 821 kB/s | 171 kB 00:00 2024-05-11T12:10:19.764 INFO:teuthology.orchestra.run.smithi050.stdout:(28/156): rbd-mirror-19.0.0-3601.g30850680.el9. 9.3 MB/s | 3.1 MB 00:00 2024-05-11T12:10:19.781 INFO:teuthology.orchestra.run.smithi050.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3601.g 292 kB/s | 34 kB 00:00 2024-05-11T12:10:19.799 INFO:teuthology.orchestra.run.smithi047.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-05-11T12:10:19.819 INFO:teuthology.orchestra.run.smithi064.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 107 kB/s | 87 kB 00:00 2024-05-11T12:10:19.823 INFO:teuthology.orchestra.run.smithi050.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3601.g3085068 2.4 MB/s | 142 kB 00:00 2024-05-11T12:10:20.011 INFO:teuthology.orchestra.run.smithi064.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 968 kB/s | 1.0 MB 00:01 2024-05-11T12:10:20.141 INFO:teuthology.orchestra.run.smithi047.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-11T12:10:20.191 INFO:teuthology.orchestra.run.smithi047.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 245 kB/s | 221 kB 00:00 2024-05-11T12:10:20.558 INFO:teuthology.orchestra.run.smithi047.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 47 kB/s | 36 kB 00:00 2024-05-11T12:10:20.628 INFO:teuthology.orchestra.run.smithi064.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 302 kB/s | 244 kB 00:00 2024-05-11T12:10:20.736 INFO:teuthology.orchestra.run.smithi064.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-05-11T12:10:20.781 INFO:teuthology.orchestra.run.smithi050.stdout:(31/156): ceph-test-19.0.0-3601.g30850680.el9.x 17 MB/s | 49 MB 00:02 2024-05-11T12:10:20.791 INFO:teuthology.orchestra.run.smithi047.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 76 kB/s | 47 kB 00:00 2024-05-11T12:10:20.828 INFO:teuthology.orchestra.run.smithi064.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 303 kB/s | 248 kB 00:00 2024-05-11T12:10:20.874 INFO:teuthology.orchestra.run.smithi050.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 7.0 MB/s | 7.4 MB 00:01 2024-05-11T12:10:20.899 INFO:teuthology.orchestra.run.smithi050.stdout:(33/156): ceph-mgr-modules-core-19.0.0-3601.g30 2.3 MB/s | 274 kB 00:00 2024-05-11T12:10:20.924 INFO:teuthology.orchestra.run.smithi050.stdout:(34/156): ceph-mgr-rook-19.0.0-3601.g30850680.e 988 kB/s | 49 kB 00:00 2024-05-11T12:10:20.949 INFO:teuthology.orchestra.run.smithi050.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3601.g3 328 kB/s | 16 kB 00:00 2024-05-11T12:10:20.974 INFO:teuthology.orchestra.run.smithi050.stdout:(36/156): ceph-volume-19.0.0-3601.g30850680.el9 5.3 MB/s | 273 kB 00:00 2024-05-11T12:10:21.016 INFO:teuthology.orchestra.run.smithi050.stdout:(37/156): cephfs-top-19.0.0-3601.g30850680.el9. 439 kB/s | 18 kB 00:00 2024-05-11T12:10:21.025 INFO:teuthology.orchestra.run.smithi047.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 366 kB/s | 305 kB 00:00 2024-05-11T12:10:21.050 INFO:teuthology.orchestra.run.smithi050.stdout:(38/156): cephadm-19.0.0-3601.g30850680.el9.noa 7.4 MB/s | 764 kB 00:00 2024-05-11T12:10:21.126 INFO:teuthology.orchestra.run.smithi050.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 740 kB/s | 80 kB 00:00 2024-05-11T12:10:21.193 INFO:teuthology.orchestra.run.smithi050.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-11T12:10:21.234 INFO:teuthology.orchestra.run.smithi047.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 86 kB/s | 58 kB 00:00 2024-05-11T12:10:21.235 INFO:teuthology.orchestra.run.smithi050.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-05-11T12:10:21.260 INFO:teuthology.orchestra.run.smithi050.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-05-11T12:10:21.270 INFO:teuthology.orchestra.run.smithi064.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 40 kB/s | 26 kB 00:00 2024-05-11T12:10:21.295 INFO:teuthology.orchestra.run.smithi064.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 34 kB/s | 19 kB 00:00 2024-05-11T12:10:21.336 INFO:teuthology.orchestra.run.smithi050.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-11T12:10:21.360 INFO:teuthology.orchestra.run.smithi050.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-11T12:10:21.386 INFO:teuthology.orchestra.run.smithi050.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-05-11T12:10:21.411 INFO:teuthology.orchestra.run.smithi050.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-11T12:10:21.436 INFO:teuthology.orchestra.run.smithi050.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-05-11T12:10:21.520 INFO:teuthology.orchestra.run.smithi050.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-05-11T12:10:21.570 INFO:teuthology.orchestra.run.smithi050.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-05-11T12:10:21.573 INFO:teuthology.orchestra.run.smithi047.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 125 kB/s | 94 kB 00:00 2024-05-11T12:10:21.596 INFO:teuthology.orchestra.run.smithi064.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 224 kB/s | 172 kB 00:00 2024-05-11T12:10:21.613 INFO:teuthology.orchestra.run.smithi050.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-05-11T12:10:21.626 INFO:teuthology.orchestra.run.smithi047.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 434 kB/s | 22 kB 00:00 2024-05-11T12:10:21.638 INFO:teuthology.orchestra.run.smithi050.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-05-11T12:10:21.688 INFO:teuthology.orchestra.run.smithi050.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 658 kB/s | 33 kB 00:00 2024-05-11T12:10:21.701 INFO:teuthology.orchestra.run.smithi047.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-05-11T12:10:21.721 INFO:teuthology.orchestra.run.smithi050.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-11T12:10:21.751 INFO:teuthology.orchestra.run.smithi047.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.7 MB/s | 863 kB 00:00 2024-05-11T12:10:21.772 INFO:teuthology.orchestra.run.smithi050.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-11T12:10:21.814 INFO:teuthology.orchestra.run.smithi050.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-05-11T12:10:21.877 INFO:teuthology.orchestra.run.smithi047.stdout:(104/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 70 kB/s | 45 kB 00:00 2024-05-11T12:10:21.879 INFO:teuthology.orchestra.run.smithi064.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-05-11T12:10:21.906 INFO:teuthology.orchestra.run.smithi050.stdout:(56/156): ceph-mgr-dashboard-19.0.0-3601.g30850 1.7 MB/s | 3.5 MB 00:02 2024-05-11T12:10:21.931 INFO:teuthology.orchestra.run.smithi050.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-11T12:10:21.982 INFO:teuthology.orchestra.run.smithi050.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-05-11T12:10:21.998 INFO:teuthology.orchestra.run.smithi050.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-05-11T12:10:22.018 INFO:teuthology.orchestra.run.smithi047.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-05-11T12:10:22.040 INFO:teuthology.orchestra.run.smithi050.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 615 kB/s | 25 kB 00:00 2024-05-11T12:10:22.052 INFO:teuthology.orchestra.run.smithi047.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-05-11T12:10:22.078 INFO:teuthology.orchestra.run.smithi047.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 947 kB/s | 309 kB 00:00 2024-05-11T12:10:22.082 INFO:teuthology.orchestra.run.smithi050.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-11T12:10:22.107 INFO:teuthology.orchestra.run.smithi050.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-05-11T12:10:22.119 INFO:teuthology.orchestra.run.smithi047.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-05-11T12:10:22.133 INFO:teuthology.orchestra.run.smithi050.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-05-11T12:10:22.145 INFO:teuthology.orchestra.run.smithi047.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-05-11T12:10:22.183 INFO:teuthology.orchestra.run.smithi050.stdout:(64/156): openblas-0.3.21-2.el9.x86_64.rpm 709 kB/s | 35 kB 00:00 2024-05-11T12:10:22.209 INFO:teuthology.orchestra.run.smithi050.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-05-11T12:10:22.234 INFO:teuthology.orchestra.run.smithi050.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-11T12:10:22.245 INFO:teuthology.orchestra.run.smithi047.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-05-11T12:10:22.270 INFO:teuthology.orchestra.run.smithi047.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 447 kB/s | 67 kB 00:00 2024-05-11T12:10:22.295 INFO:teuthology.orchestra.run.smithi047.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-05-11T12:10:22.329 INFO:teuthology.orchestra.run.smithi047.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-05-11T12:10:22.363 INFO:teuthology.orchestra.run.smithi047.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-05-11T12:10:22.384 INFO:teuthology.orchestra.run.smithi050.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-11T12:10:22.396 INFO:teuthology.orchestra.run.smithi047.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 481 kB/s | 32 kB 00:00 2024-05-11T12:10:22.421 INFO:teuthology.orchestra.run.smithi064.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 304 kB/s | 250 kB 00:00 2024-05-11T12:10:22.430 INFO:teuthology.orchestra.run.smithi047.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-05-11T12:10:22.464 INFO:teuthology.orchestra.run.smithi047.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-11T12:10:22.496 INFO:teuthology.orchestra.run.smithi064.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-05-11T12:10:22.509 INFO:teuthology.orchestra.run.smithi050.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 15 MB/s | 4.7 MB 00:00 2024-05-11T12:10:22.531 INFO:teuthology.orchestra.run.smithi047.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-05-11T12:10:22.556 INFO:teuthology.orchestra.run.smithi047.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-11T12:10:22.581 INFO:teuthology.orchestra.run.smithi047.stdout:(120/156): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-05-11T12:10:22.606 INFO:teuthology.orchestra.run.smithi047.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 469 kB/s | 35 kB 00:00 2024-05-11T12:10:22.632 INFO:teuthology.orchestra.run.smithi047.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-11T12:10:22.635 INFO:teuthology.orchestra.run.smithi050.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-11T12:10:22.674 INFO:teuthology.orchestra.run.smithi047.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-11T12:10:22.677 INFO:teuthology.orchestra.run.smithi050.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 3.0 MB/s | 911 kB 00:00 2024-05-11T12:10:22.691 INFO:teuthology.orchestra.run.smithi047.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-05-11T12:10:22.714 INFO:teuthology.orchestra.run.smithi064.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 266 kB/s | 222 kB 00:00 2024-05-11T12:10:22.719 INFO:teuthology.orchestra.run.smithi050.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 638 kB/s | 133 kB 00:00 2024-05-11T12:10:22.733 INFO:teuthology.orchestra.run.smithi047.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 303 kB/s | 18 kB 00:00 2024-05-11T12:10:22.750 INFO:teuthology.orchestra.run.smithi047.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 394 kB/s | 23 kB 00:00 2024-05-11T12:10:22.777 INFO:teuthology.orchestra.run.smithi050.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-11T12:10:22.791 INFO:teuthology.orchestra.run.smithi047.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-05-11T12:10:22.808 INFO:teuthology.orchestra.run.smithi047.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-05-11T12:10:22.827 INFO:teuthology.orchestra.run.smithi050.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-05-11T12:10:22.867 INFO:teuthology.orchestra.run.smithi047.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-05-11T12:10:22.892 INFO:teuthology.orchestra.run.smithi047.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-05-11T12:10:22.911 INFO:teuthology.orchestra.run.smithi050.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-11T12:10:22.934 INFO:teuthology.orchestra.run.smithi047.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.2 MB/s | 79 kB 00:00 2024-05-11T12:10:22.960 INFO:teuthology.orchestra.run.smithi047.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-05-11T12:10:22.995 INFO:teuthology.orchestra.run.smithi050.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-11T12:10:23.002 INFO:teuthology.orchestra.run.smithi047.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-11T12:10:23.027 INFO:teuthology.orchestra.run.smithi047.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-11T12:10:23.068 INFO:teuthology.orchestra.run.smithi047.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-05-11T12:10:23.087 INFO:teuthology.orchestra.run.smithi050.stdout:(76/156): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-05-11T12:10:23.095 INFO:teuthology.orchestra.run.smithi047.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 419 kB/s | 28 kB 00:00 2024-05-11T12:10:23.105 INFO:teuthology.orchestra.run.smithi064.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 113 kB/s | 77 kB 00:00 2024-05-11T12:10:23.112 INFO:teuthology.orchestra.run.smithi050.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-05-11T12:10:23.136 INFO:teuthology.orchestra.run.smithi047.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-05-11T12:10:23.155 INFO:teuthology.orchestra.run.smithi064.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 64 kB/s | 42 kB 00:00 2024-05-11T12:10:23.162 INFO:teuthology.orchestra.run.smithi047.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-05-11T12:10:23.196 INFO:teuthology.orchestra.run.smithi050.stdout:(78/156): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-05-11T12:10:23.203 INFO:teuthology.orchestra.run.smithi047.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 883 kB/s | 59 kB 00:00 2024-05-11T12:10:23.221 INFO:teuthology.orchestra.run.smithi050.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-11T12:10:23.229 INFO:teuthology.orchestra.run.smithi047.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-05-11T12:10:23.271 INFO:teuthology.orchestra.run.smithi050.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 463 kB/s | 35 kB 00:00 2024-05-11T12:10:23.279 INFO:teuthology.orchestra.run.smithi047.stdout:(141/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-05-11T12:10:23.288 INFO:teuthology.orchestra.run.smithi050.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 979 kB/s | 172 kB 00:00 2024-05-11T12:10:23.304 INFO:teuthology.orchestra.run.smithi047.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 507 kB/s | 51 kB 00:00 2024-05-11T12:10:23.330 INFO:teuthology.orchestra.run.smithi047.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-05-11T12:10:23.355 INFO:teuthology.orchestra.run.smithi047.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-11T12:10:23.363 INFO:teuthology.orchestra.run.smithi050.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-11T12:10:23.380 INFO:teuthology.orchestra.run.smithi047.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-05-11T12:10:23.389 INFO:teuthology.orchestra.run.smithi050.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-05-11T12:10:23.405 INFO:teuthology.orchestra.run.smithi047.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 264 kB/s | 20 kB 00:00 2024-05-11T12:10:23.431 INFO:teuthology.orchestra.run.smithi047.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-11T12:10:23.447 INFO:teuthology.orchestra.run.smithi050.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 921 kB/s | 77 kB 00:00 2024-05-11T12:10:23.473 INFO:teuthology.orchestra.run.smithi047.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-11T12:10:23.473 INFO:teuthology.orchestra.run.smithi050.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 505 kB/s | 42 kB 00:00 2024-05-11T12:10:23.530 INFO:teuthology.orchestra.run.smithi064.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 330 kB/s | 269 kB 00:00 2024-05-11T12:10:23.548 INFO:teuthology.orchestra.run.smithi050.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-05-11T12:10:23.590 INFO:teuthology.orchestra.run.smithi050.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-05-11T12:10:23.615 INFO:teuthology.orchestra.run.smithi050.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-05-11T12:10:23.648 INFO:teuthology.orchestra.run.smithi047.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-05-11T12:10:23.665 INFO:teuthology.orchestra.run.smithi050.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-05-11T12:10:23.691 INFO:teuthology.orchestra.run.smithi050.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-05-11T12:10:23.717 INFO:teuthology.orchestra.run.smithi050.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 533 kB/s | 54 kB 00:00 2024-05-11T12:10:23.766 INFO:teuthology.orchestra.run.smithi050.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-05-11T12:10:23.792 INFO:teuthology.orchestra.run.smithi050.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-05-11T12:10:23.831 INFO:teuthology.orchestra.run.smithi047.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 10 MB/s | 1.8 MB 00:00 2024-05-11T12:10:23.842 INFO:teuthology.orchestra.run.smithi050.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-05-11T12:10:23.867 INFO:teuthology.orchestra.run.smithi050.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-11T12:10:23.890 INFO:teuthology.orchestra.run.smithi047.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 717 kB/s | 42 kB 00:00 2024-05-11T12:10:23.906 INFO:teuthology.orchestra.run.smithi064.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 198 kB/s | 159 kB 00:00 2024-05-11T12:10:23.918 INFO:teuthology.orchestra.run.smithi050.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-05-11T12:10:23.940 INFO:teuthology.orchestra.run.smithi047.stdout:(152/156): librados2-19.0.0-3601.g30850680.el9. 6.4 MB/s | 3.4 MB 00:00 2024-05-11T12:10:23.968 INFO:teuthology.orchestra.run.smithi050.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-11T12:10:23.989 INFO:teuthology.orchestra.run.smithi064.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 334 kB/s | 279 kB 00:00 2024-05-11T12:10:23.994 INFO:teuthology.orchestra.run.smithi050.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-05-11T12:10:24.007 INFO:teuthology.orchestra.run.smithi047.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-05-11T12:10:24.052 INFO:teuthology.orchestra.run.smithi050.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-11T12:10:24.057 INFO:teuthology.orchestra.run.smithi047.stdout:(154/156): librbd1-19.0.0-3601.g30850680.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-05-11T12:10:24.077 INFO:teuthology.orchestra.run.smithi050.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-05-11T12:10:24.136 INFO:teuthology.orchestra.run.smithi050.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-05-11T12:10:24.161 INFO:teuthology.orchestra.run.smithi050.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-05-11T12:10:24.190 INFO:teuthology.orchestra.run.smithi064.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 77 kB/s | 51 kB 00:00 2024-05-11T12:10:24.191 INFO:teuthology.orchestra.run.smithi047.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 5.4 MB/s | 1.3 MB 00:00 2024-05-11T12:10:24.317 INFO:teuthology.orchestra.run.smithi047.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 2.4 MB 00:00 2024-05-11T12:10:24.321 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-05-11T12:10:24.321 INFO:teuthology.orchestra.run.smithi047.stdout:Total 8.9 MB/s | 226 MB 00:25 2024-05-11T12:10:24.403 INFO:teuthology.orchestra.run.smithi050.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-11T12:10:24.479 INFO:teuthology.orchestra.run.smithi050.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-05-11T12:10:24.604 INFO:teuthology.orchestra.run.smithi050.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-05-11T12:10:24.615 INFO:teuthology.orchestra.run.smithi064.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 76 kB/s | 54 kB 00:00 2024-05-11T12:10:24.654 INFO:teuthology.orchestra.run.smithi050.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-11T12:10:24.713 INFO:teuthology.orchestra.run.smithi050.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-05-11T12:10:24.771 INFO:teuthology.orchestra.run.smithi050.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 836 kB/s | 49 kB 00:00 2024-05-11T12:10:24.805 INFO:teuthology.orchestra.run.smithi050.stdout:(109/156): abseil-cpp-20211102.0-3.el9.x86_64.r 852 kB/s | 548 kB 00:00 2024-05-11T12:10:24.822 INFO:teuthology.orchestra.run.smithi050.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-11T12:10:24.832 INFO:teuthology.orchestra.run.smithi064.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 65 kB/s | 42 kB 00:00 2024-05-11T12:10:24.930 INFO:teuthology.orchestra.run.smithi050.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-05-11T12:10:24.988 INFO:teuthology.orchestra.run.smithi050.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.0 MB/s | 839 kB 00:00 2024-05-11T12:10:25.065 INFO:teuthology.orchestra.run.smithi050.stdout:(113/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-05-11T12:10:25.106 INFO:teuthology.orchestra.run.smithi050.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 367 kB/s | 43 kB 00:00 2024-05-11T12:10:25.132 INFO:teuthology.orchestra.run.smithi050.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-05-11T12:10:25.157 INFO:teuthology.orchestra.run.smithi050.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-05-11T12:10:25.190 INFO:teuthology.orchestra.run.smithi050.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-11T12:10:25.224 INFO:teuthology.orchestra.run.smithi050.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-05-11T12:10:25.241 INFO:teuthology.orchestra.run.smithi064.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 71 kB/s | 44 kB 00:00 2024-05-11T12:10:25.250 INFO:teuthology.orchestra.run.smithi050.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-05-11T12:10:25.275 INFO:teuthology.orchestra.run.smithi050.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-11T12:10:25.300 INFO:teuthology.orchestra.run.smithi050.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 466 kB/s | 35 kB 00:00 2024-05-11T12:10:25.325 INFO:teuthology.orchestra.run.smithi050.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-11T12:10:25.384 INFO:teuthology.orchestra.run.smithi050.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-05-11T12:10:25.410 INFO:teuthology.orchestra.run.smithi050.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 127 kB/s | 11 kB 00:00 2024-05-11T12:10:25.435 INFO:teuthology.orchestra.run.smithi050.stdout:(125/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-11T12:10:25.452 INFO:teuthology.orchestra.run.smithi050.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 262 kB/s | 18 kB 00:00 2024-05-11T12:10:25.469 INFO:teuthology.orchestra.run.smithi050.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 393 kB/s | 23 kB 00:00 2024-05-11T12:10:25.494 INFO:teuthology.orchestra.run.smithi050.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 330 kB/s | 19 kB 00:00 2024-05-11T12:10:25.511 INFO:teuthology.orchestra.run.smithi050.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-05-11T12:10:25.526 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-05-11T12:10:25.544 INFO:teuthology.orchestra.run.smithi050.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-05-11T12:10:25.570 INFO:teuthology.orchestra.run.smithi050.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 612 kB/s | 46 kB 00:00 2024-05-11T12:10:25.574 INFO:teuthology.orchestra.run.smithi064.stdout:(94/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-11T12:10:25.595 INFO:teuthology.orchestra.run.smithi050.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-05-11T12:10:25.620 INFO:teuthology.orchestra.run.smithi050.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-05-11T12:10:25.654 INFO:teuthology.orchestra.run.smithi050.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-05-11T12:10:25.679 INFO:teuthology.orchestra.run.smithi050.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-11T12:10:25.697 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-05-11T12:10:25.697 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-05-11T12:10:25.705 INFO:teuthology.orchestra.run.smithi050.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-05-11T12:10:25.730 INFO:teuthology.orchestra.run.smithi050.stdout:(137/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-05-11T12:10:25.755 INFO:teuthology.orchestra.run.smithi050.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-11T12:10:25.800 INFO:teuthology.orchestra.run.smithi064.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 228 kB/s | 221 kB 00:00 2024-05-11T12:10:25.806 INFO:teuthology.orchestra.run.smithi050.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 785 kB/s | 59 kB 00:00 2024-05-11T12:10:25.831 INFO:teuthology.orchestra.run.smithi050.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-05-11T12:10:25.856 INFO:teuthology.orchestra.run.smithi050.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 355 kB/s | 36 kB 00:00 2024-05-11T12:10:25.881 INFO:teuthology.orchestra.run.smithi050.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-05-11T12:10:25.907 INFO:teuthology.orchestra.run.smithi050.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-11T12:10:25.932 INFO:teuthology.orchestra.run.smithi050.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-11T12:10:25.966 INFO:teuthology.orchestra.run.smithi050.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-05-11T12:10:25.983 INFO:teuthology.orchestra.run.smithi064.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 48 kB/s | 36 kB 00:00 2024-05-11T12:10:26.000 INFO:teuthology.orchestra.run.smithi050.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-11T12:10:26.091 INFO:teuthology.orchestra.run.smithi050.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-05-11T12:10:26.133 INFO:teuthology.orchestra.run.smithi050.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-11T12:10:26.284 INFO:teuthology.orchestra.run.smithi064.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 69 kB/s | 47 kB 00:00 2024-05-11T12:10:26.383 INFO:teuthology.orchestra.run.smithi050.stdout:(149/156): librbd1-19.0.0-3601.g30850680.el9.x8 11 MB/s | 3.1 MB 00:00 2024-05-11T12:10:26.667 INFO:teuthology.orchestra.run.smithi064.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 85 kB/s | 58 kB 00:00 2024-05-11T12:10:26.700 INFO:teuthology.orchestra.run.smithi064.stdout:(99/156): socat-1.7.4.1-5.el9.x86_64.rpm 338 kB/s | 305 kB 00:00 2024-05-11T12:10:26.826 INFO:teuthology.orchestra.run.smithi050.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 3.3 MB/s | 2.3 MB 00:00 2024-05-11T12:10:26.867 INFO:teuthology.orchestra.run.smithi050.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-11T12:10:26.943 INFO:teuthology.orchestra.run.smithi050.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.3 MB/s | 1.8 MB 00:00 2024-05-11T12:10:26.968 INFO:teuthology.orchestra.run.smithi050.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.4 MB/s | 248 kB 00:00 2024-05-11T12:10:27.017 INFO:teuthology.orchestra.run.smithi064.stdout:(100/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 128 kB/s | 94 kB 00:00 2024-05-11T12:10:27.101 INFO:teuthology.orchestra.run.smithi064.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 272 kB/s | 22 kB 00:00 2024-05-11T12:10:27.152 INFO:teuthology.orchestra.run.smithi050.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-05-11T12:10:27.226 INFO:teuthology.orchestra.run.smithi064.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 6.8 MB/s | 863 kB 00:00 2024-05-11T12:10:27.260 INFO:teuthology.orchestra.run.smithi050.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.2 MB/s | 2.4 MB 00:00 2024-05-11T12:10:27.335 INFO:teuthology.orchestra.run.smithi064.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-05-11T12:10:27.344 INFO:teuthology.orchestra.run.smithi050.stdout:(156/156): librados2-19.0.0-3601.g30850680.el9. 2.5 MB/s | 3.4 MB 00:01 2024-05-11T12:10:27.347 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-11T12:10:27.347 INFO:teuthology.orchestra.run.smithi050.stdout:Total 18 MB/s | 226 MB 00:12 2024-05-11T12:10:27.360 INFO:teuthology.orchestra.run.smithi064.stdout:(104/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-05-11T12:10:27.511 INFO:teuthology.orchestra.run.smithi064.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-05-11T12:10:27.577 INFO:teuthology.orchestra.run.smithi064.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-05-11T12:10:27.603 INFO:teuthology.orchestra.run.smithi064.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-05-11T12:10:27.645 INFO:teuthology.orchestra.run.smithi064.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-05-11T12:10:27.670 INFO:teuthology.orchestra.run.smithi064.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-05-11T12:10:27.712 INFO:teuthology.orchestra.run.smithi064.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-11T12:10:27.737 INFO:teuthology.orchestra.run.smithi064.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-05-11T12:10:27.837 INFO:teuthology.orchestra.run.smithi064.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 839 kB 00:00 2024-05-11T12:10:27.862 INFO:teuthology.orchestra.run.smithi064.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-11T12:10:27.896 INFO:teuthology.orchestra.run.smithi064.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-05-11T12:10:27.982 INFO:teuthology.orchestra.run.smithi064.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-05-11T12:10:27.998 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-05-11T12:10:27.999 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-05-11T12:10:28.005 INFO:teuthology.orchestra.run.smithi064.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-05-11T12:10:28.022 INFO:teuthology.orchestra.run.smithi064.stdout:(117/156): python3-cachetools-4.2.4-1.el9.noarc 202 kB/s | 32 kB 00:00 2024-05-11T12:10:28.047 INFO:teuthology.orchestra.run.smithi064.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.6 MB/s | 172 kB 00:00 2024-05-11T12:10:28.073 INFO:teuthology.orchestra.run.smithi064.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-11T12:10:28.114 INFO:teuthology.orchestra.run.smithi064.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 526 kB/s | 35 kB 00:00 2024-05-11T12:10:28.140 INFO:teuthology.orchestra.run.smithi064.stdout:(121/156): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-11T12:10:28.165 INFO:teuthology.orchestra.run.smithi064.stdout:(122/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-05-11T12:10:28.207 INFO:teuthology.orchestra.run.smithi064.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-11T12:10:28.232 INFO:teuthology.orchestra.run.smithi064.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-05-11T12:10:28.283 INFO:teuthology.orchestra.run.smithi064.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-05-11T12:10:28.308 INFO:teuthology.orchestra.run.smithi064.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-05-11T12:10:28.333 INFO:teuthology.orchestra.run.smithi064.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-11T12:10:28.358 INFO:teuthology.orchestra.run.smithi064.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-11T12:10:28.400 INFO:teuthology.orchestra.run.smithi064.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-05-11T12:10:28.426 INFO:teuthology.orchestra.run.smithi064.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-05-11T12:10:28.459 INFO:teuthology.orchestra.run.smithi064.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-05-11T12:10:28.484 INFO:teuthology.orchestra.run.smithi064.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 942 kB/s | 79 kB 00:00 2024-05-11T12:10:28.510 INFO:teuthology.orchestra.run.smithi064.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-05-11T12:10:28.535 INFO:teuthology.orchestra.run.smithi064.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-11T12:10:28.560 INFO:teuthology.orchestra.run.smithi064.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-11T12:10:28.565 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-11T12:10:28.586 INFO:teuthology.orchestra.run.smithi064.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-05-11T12:10:28.611 INFO:teuthology.orchestra.run.smithi064.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-05-11T12:10:28.636 INFO:teuthology.orchestra.run.smithi064.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-05-11T12:10:28.661 INFO:teuthology.orchestra.run.smithi064.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-05-11T12:10:28.687 INFO:teuthology.orchestra.run.smithi064.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 786 kB/s | 59 kB 00:00 2024-05-11T12:10:28.712 INFO:teuthology.orchestra.run.smithi064.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-11T12:10:28.734 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-11T12:10:28.734 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-11T12:10:28.737 INFO:teuthology.orchestra.run.smithi064.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-05-11T12:10:28.765 INFO:teuthology.orchestra.run.smithi064.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-11T12:10:28.779 INFO:teuthology.orchestra.run.smithi064.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-11T12:10:28.813 INFO:teuthology.orchestra.run.smithi064.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-05-11T12:10:28.838 INFO:teuthology.orchestra.run.smithi064.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-05-11T12:10:28.863 INFO:teuthology.orchestra.run.smithi064.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-05-11T12:10:28.905 INFO:teuthology.orchestra.run.smithi064.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-11T12:10:29.106 INFO:teuthology.orchestra.run.smithi064.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-05-11T12:10:29.198 INFO:teuthology.orchestra.run.smithi064.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-05-11T12:10:29.240 INFO:teuthology.orchestra.run.smithi064.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-05-11T12:10:29.290 INFO:teuthology.orchestra.run.smithi064.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.9 MB/s | 248 kB 00:00 2024-05-11T12:10:29.365 INFO:teuthology.orchestra.run.smithi064.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-05-11T12:10:29.482 INFO:teuthology.orchestra.run.smithi064.stdout:(154/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-05-11T12:10:29.558 INFO:teuthology.orchestra.run.smithi064.stdout:(155/156): librbd1-19.0.0-3601.g30850680.el9.x8 4.5 MB/s | 3.1 MB 00:00 2024-05-11T12:10:29.808 INFO:teuthology.orchestra.run.smithi064.stdout:(156/156): librados2-19.0.0-3601.g30850680.el9. 3.5 MB/s | 3.4 MB 00:00 2024-05-11T12:10:29.811 INFO:teuthology.orchestra.run.smithi064.stdout:-------------------------------------------------------------------------------- 2024-05-11T12:10:29.811 INFO:teuthology.orchestra.run.smithi064.stdout:Total 7.8 MB/s | 226 MB 00:29 2024-05-11T12:10:30.656 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-11T12:10:30.656 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-11T12:10:31.035 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction check 2024-05-11T12:10:31.207 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction check succeeded. 2024-05-11T12:10:31.208 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction test 2024-05-11T12:10:33.151 INFO:teuthology.orchestra.run.smithi064.stdout:Transaction test succeeded. 2024-05-11T12:10:33.151 INFO:teuthology.orchestra.run.smithi064.stdout:Running transaction 2024-05-11T12:10:34.432 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-05-11T12:10:34.530 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T12:10:34.575 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T12:10:34.651 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T12:10:34.690 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T12:10:34.714 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T12:10:34.776 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T12:10:35.128 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T12:10:35.232 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:10:35.348 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:10:35.374 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:10:35.458 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:10:35.487 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:10:35.667 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T12:10:35.718 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T12:10:35.745 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:10:35.850 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:10:35.879 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:10:35.943 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:10:35.972 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:10:35.998 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:10:36.217 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:10:36.388 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:10:36.520 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:10:36.580 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:10:36.686 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:36.756 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:36.968 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:36.997 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-11T12:10:37.073 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T12:10:37.074 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T12:10:37.102 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T12:10:37.129 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T12:10:37.137 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T12:10:37.183 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T12:10:37.221 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T12:10:37.225 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T12:10:37.250 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T12:10:37.261 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T12:10:37.293 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T12:10:37.305 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T12:10:37.338 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T12:10:37.365 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T12:10:37.436 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T12:10:37.488 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T12:10:37.518 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T12:10:37.562 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T12:10:37.640 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T12:10:37.641 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T12:10:37.672 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T12:10:37.716 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T12:10:37.742 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:10:37.744 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T12:10:37.817 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T12:10:37.867 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:10:37.868 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T12:10:37.902 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:10:37.946 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T12:10:37.995 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:10:38.023 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:10:38.090 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T12:10:38.146 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T12:10:38.180 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T12:10:38.237 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T12:10:38.264 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:10:38.371 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:10:38.399 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:10:38.445 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:10:38.475 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:10:38.502 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:10:38.693 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:10:38.700 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T12:10:38.784 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T12:10:38.819 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T12:10:38.822 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:10:38.862 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T12:10:38.900 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T12:10:38.949 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:10:38.953 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T12:10:39.018 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:10:39.111 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T12:10:39.119 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:39.178 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T12:10:39.194 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:39.272 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T12:10:39.335 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T12:10:39.395 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:39.409 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T12:10:39.426 INFO:teuthology.orchestra.run.smithi064.stdout: Preparing : 1/1 2024-05-11T12:10:39.446 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T12:10:39.499 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T12:10:39.505 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T12:10:39.521 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-11T12:10:39.532 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T12:10:39.547 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T12:10:39.559 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-11T12:10:39.582 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T12:10:39.592 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T12:10:39.622 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-11T12:10:39.626 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T12:10:39.663 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-11T12:10:39.667 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T12:10:39.676 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T12:10:39.688 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-11T12:10:39.716 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T12:10:39.732 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-11T12:10:39.760 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T12:10:39.792 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T12:10:39.819 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T12:10:39.820 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T12:10:39.864 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T12:10:39.894 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T12:10:39.900 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T12:10:39.937 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T12:10:39.952 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T12:10:39.982 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T12:10:40.026 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T12:10:40.034 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T12:10:40.070 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T12:10:40.078 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-11T12:10:40.103 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T12:10:40.136 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T12:10:40.136 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T12:10:40.168 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T12:10:40.180 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T12:10:40.192 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T12:10:40.206 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:10:40.208 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T12:10:40.280 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T12:10:40.289 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T12:10:40.314 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:10:40.323 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T12:10:40.341 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:10:40.402 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T12:10:40.417 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:10:40.424 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T12:10:40.445 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:10:40.458 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T12:10:40.486 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T12:10:40.521 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T12:10:40.544 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T12:10:40.560 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T12:10:40.593 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T12:10:40.600 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T12:10:40.604 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-11T12:10:40.630 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T12:10:40.659 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-11T12:10:40.664 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T12:10:40.686 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:10:40.697 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T12:10:40.739 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T12:10:40.773 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T12:10:40.791 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:10:40.829 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:10:40.863 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T12:10:40.892 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 14/164 2024-05-11T12:10:40.913 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:10:40.922 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T12:10:40.934 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-11T12:10:40.960 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T12:10:41.016 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T12:10:41.127 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T12:10:41.131 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:10:41.146 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T12:10:41.187 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T12:10:41.233 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T12:10:41.261 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-11T12:10:41.267 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T12:10:41.310 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T12:10:41.347 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T12:10:41.382 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T12:10:41.429 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:10:41.471 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-11T12:10:41.534 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T12:10:41.578 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:41.601 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T12:10:41.647 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:41.695 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T12:10:41.758 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T12:10:41.833 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T12:10:41.841 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-11T12:10:41.869 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T12:10:41.905 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T12:10:41.951 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T12:10:41.953 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T12:10:41.983 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-11T12:10:41.988 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T12:10:41.992 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T12:10:42.010 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-11T12:10:42.025 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T12:10:42.040 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T12:10:42.046 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-11T12:10:42.082 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T12:10:42.122 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-11T12:10:42.128 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T12:10:42.162 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-11T12:10:42.193 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T12:10:42.206 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-11T12:10:42.225 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T12:10:42.246 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-11T12:10:42.271 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T12:10:42.273 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-11T12:10:42.291 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T12:10:42.301 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T12:10:42.327 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T12:10:42.343 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T12:10:42.346 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-11T12:10:42.397 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-11T12:10:42.426 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-11T12:10:42.440 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T12:10:42.470 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-11T12:10:42.476 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T12:10:42.512 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T12:10:42.542 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T12:10:42.549 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-11T12:10:42.581 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 32/164 2024-05-11T12:10:42.610 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T12:10:42.623 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T12:10:42.625 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 33/164 2024-05-11T12:10:42.653 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 34/164 2024-05-11T12:10:42.728 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-11T12:10:42.776 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-11T12:10:42.783 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T12:10:42.808 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T12:10:42.856 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-11T12:10:42.903 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T12:10:43.009 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-11T12:10:43.035 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T12:10:43.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-11T12:10:43.082 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T12:10:43.118 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T12:10:43.152 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T12:10:43.192 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T12:10:43.265 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T12:10:43.295 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T12:10:43.308 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T12:10:43.330 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T12:10:43.363 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T12:10:43.405 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T12:10:43.427 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T12:10:43.439 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T12:10:43.529 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T12:10:43.578 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T12:10:43.609 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T12:10:43.627 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-11T12:10:43.649 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T12:10:43.719 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-11T12:10:43.753 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-11T12:10:43.759 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T12:10:43.796 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-11T12:10:43.811 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T12:10:43.834 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 44/164 2024-05-11T12:10:43.877 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-11T12:10:44.049 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-11T12:10:44.112 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-11T12:10:44.209 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-11T12:10:44.269 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-11T12:10:44.343 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-11T12:10:44.379 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-11T12:10:44.416 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-11T12:10:44.465 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-11T12:10:44.499 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-11T12:10:44.542 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-11T12:10:44.570 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T12:10:44.584 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-11T12:10:44.624 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T12:10:44.658 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T12:10:44.756 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-11T12:10:44.760 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T12:10:44.769 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T12:10:44.807 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-11T12:10:44.817 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T12:10:44.836 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-11T12:10:44.837 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T12:10:44.865 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T12:10:44.878 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-11T12:10:44.898 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T12:10:44.906 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T12:10:44.943 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T12:10:44.977 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-11T12:10:45.011 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-11T12:10:45.051 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-11T12:10:45.093 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-11T12:10:45.117 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-11T12:10:45.130 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T12:10:45.158 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T12:10:45.197 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-11T12:10:45.233 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T12:10:45.239 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T12:10:45.277 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:10:45.326 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-11T12:10:45.361 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:10:45.366 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-11T12:10:45.391 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T12:10:45.394 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-11T12:10:45.429 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-11T12:10:45.435 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T12:10:45.468 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-11T12:10:45.508 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-11T12:10:45.546 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-11T12:10:45.581 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-11T12:10:45.614 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-11T12:10:45.656 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-11T12:10:45.690 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-11T12:10:45.781 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-11T12:10:45.829 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-11T12:10:45.860 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-11T12:10:45.907 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-11T12:10:45.914 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T12:10:45.951 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T12:10:45.989 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:10:46.020 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-11T12:10:46.037 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T12:10:46.044 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:10:46.071 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-11T12:10:46.073 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T12:10:46.851 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-11T12:10:46.900 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-11T12:10:46.933 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-11T12:10:47.037 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-11T12:10:47.092 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-11T12:10:47.185 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-11T12:10:47.218 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-11T12:10:47.328 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T12:10:47.406 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-11T12:10:47.406 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T12:10:47.432 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-11T12:10:47.434 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T12:10:47.468 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T12:10:47.514 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-11T12:10:47.797 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T12:10:47.839 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:10:47.931 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:10:47.961 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T12:10:47.997 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T12:10:48.209 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-11T12:10:48.319 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-11T12:10:48.488 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:48.505 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T12:10:48.528 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:48.534 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:10:48.589 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:10:48.619 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T12:10:49.126 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:49.165 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:10:49.259 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:10:49.259 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T12:10:49.259 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T12:10:49.259 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:10:49.299 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:10:49.650 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-11T12:10:49.719 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-11T12:10:49.747 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-11T12:10:49.780 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-11T12:10:50.110 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-11T12:10:50.143 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:10:50.235 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 101/164 2024-05-11T12:10:50.265 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 102/164 2024-05-11T12:10:50.301 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-11T12:10:50.810 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-11T12:10:50.837 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:10:50.901 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 105/164 2024-05-11T12:10:50.930 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 106/164 2024-05-11T12:10:51.037 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:51.074 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:51.646 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:51.686 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:10:51.777 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:10:51.777 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T12:10:51.777 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T12:10:51.777 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:10:51.804 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:10:53.766 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:53.802 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:54.378 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 107/164 2024-05-11T12:10:54.413 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:10:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 108/164 2024-05-11T12:10:54.506 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-11T12:10:54.506 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-11T12:10:54.506 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:10:54.540 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:11:00.992 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:11:00.992 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-05-11T12:11:00.992 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-05-11T12:11:00.992 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-05-11T12:11:00.992 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-05-11T12:11:00.993 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-05-11T12:11:00.993 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-05-11T12:11:00.993 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-05-11T12:11:00.993 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:01.172 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:11:01.189 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:11:01.190 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:01.190 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T12:11:01.190 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:02.733 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:11:02.761 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:11:02.761 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:02.761 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T12:11:02.761 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:11:02.761 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:11:02.761 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:02.992 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:11:03.017 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:11:03.017 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:03.017 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T12:11:03.017 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:11:03.017 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:11:03.017 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:03.484 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:11:03.509 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:11:03.509 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:03.509 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T12:11:03.509 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:11:03.509 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:11:03.509 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:03.615 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T12:11:03.673 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-05-11T12:11:03.681 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:03.737 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T12:11:03.853 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:11:03.869 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:11:03.869 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:03.870 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T12:11:03.870 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:03.933 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T12:11:03.986 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T12:11:04.067 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T12:11:04.164 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T12:11:04.315 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T12:11:04.378 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T12:11:04.409 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T12:11:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T12:11:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:11:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:11:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T12:11:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T12:11:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:11:05.408 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:11:05.436 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:11:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T12:11:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:11:05.436 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:11:05.436 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:05.675 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:11:05.700 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:11:05.700 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:05.700 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T12:11:05.700 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:11:05.700 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:11:05.701 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:06.064 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:11:06.101 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:11:06.159 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:11:06.183 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:11:06.184 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:06.184 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T12:11:06.184 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:11:06.184 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:11:06.184 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:06.290 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T12:11:06.348 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T12:11:06.412 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T12:11:06.505 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 109/164 2024-05-11T12:11:06.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /sys 2024-05-11T12:11:06.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /proc 2024-05-11T12:11:06.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /mnt 2024-05-11T12:11:06.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /var/tmp 2024-05-11T12:11:06.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /home 2024-05-11T12:11:06.505 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /root 2024-05-11T12:11:06.506 INFO:teuthology.orchestra.run.smithi064.stdout:skipping the directory /tmp 2024-05-11T12:11:06.506 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:06.616 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T12:11:06.670 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T12:11:06.688 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:11:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 110/164 2024-05-11T12:11:06.705 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:06.705 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-11T12:11:06.705 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:06.750 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T12:11:06.840 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T12:11:06.981 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T12:11:07.028 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T12:11:07.060 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T12:11:07.087 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T12:11:07.118 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:11:07.144 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:11:07.261 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:11:07.496 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T12:11:07.499 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:11:07.520 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:11:07.528 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:11:07.545 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:11:07.545 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:07.546 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T12:11:07.546 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:11:07.546 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:11:07.546 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:07.560 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T12:11:07.596 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:11:07.607 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:11:07.624 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T12:11:07.651 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:11:07.729 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T12:11:07.792 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T12:11:08.266 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:11:08.294 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 111/164 2024-05-11T12:11:08.294 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:08.295 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-11T12:11:08.295 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:11:08.295 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-11T12:11:08.295 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:08.534 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:11:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 112/164 2024-05-11T12:11:08.558 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:08.558 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-11T12:11:08.558 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:11:08.558 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-11T12:11:08.558 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:08.755 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:11:08.775 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T12:11:08.786 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:11:08.807 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:11:08.832 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:11:08.832 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:08.832 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T12:11:08.832 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:11:08.832 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:11:08.832 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:08.874 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:11:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:11:08.898 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:08.898 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T12:11:08.898 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:08.958 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:11:08.982 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:11:08.982 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:08.983 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T12:11:08.983 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:11:08.983 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:11:08.983 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:09.017 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:11:09.041 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 113/164 2024-05-11T12:11:09.041 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:09.041 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-11T12:11:09.041 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:11:09.042 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-11T12:11:09.042 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:09.149 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-11T12:11:09.207 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-11T12:11:09.271 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-11T12:11:09.300 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:11:09.325 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:11:09.325 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:09.325 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T12:11:09.325 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:11:09.325 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:11:09.325 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:09.468 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-11T12:11:09.519 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-11T12:11:09.601 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-11T12:11:09.699 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-11T12:11:09.807 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:11:09.836 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:11:09.851 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-11T12:11:09.894 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-11T12:11:09.926 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-11T12:11:09.953 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-11T12:11:09.956 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:11:10.131 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T12:11:10.164 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:11:10.189 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:11:10.189 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:10.189 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T12:11:10.189 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:11:10.189 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:11:10.189 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:10.240 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:11:10.251 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:11:10.353 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T12:11:10.406 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T12:11:10.414 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:11:10.443 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephadm-2:19.0.0-3601.g30850680.el9.noarch 125/164 2024-05-11T12:11:10.478 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 126/164 2024-05-11T12:11:10.539 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 127/164 2024-05-11T12:11:10.566 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:11:11.401 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T12:11:11.434 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:11:11.459 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:11:11.459 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:11.459 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T12:11:11.459 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:11:11.459 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:11:11.459 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:11.501 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:11:11.525 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:11:11.525 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:11.525 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T12:11:11.525 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:11.577 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:11:11.601 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:11:11.601 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:11.601 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T12:11:11.601 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:11:11.601 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:11:11.602 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:11.674 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 128/164 2024-05-11T12:11:11.709 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:11:11.919 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:11:11.944 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:11:11.944 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:11.944 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T12:11:11.944 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:11:11.944 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:11:11.944 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:12.730 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 129/164 2024-05-11T12:11:12.777 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:11:12.898 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 130/164 2024-05-11T12:11:13.063 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 131/164 2024-05-11T12:11:13.179 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:11:13.203 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 132/164 2024-05-11T12:11:13.204 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:13.204 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-11T12:11:13.204 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:11:13.204 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-11T12:11:13.204 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:13.254 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:11:13.266 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 133/164 2024-05-11T12:11:13.368 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-2:19.0.0-3601.g30850680.el9.x86_64 134/164 2024-05-11T12:11:13.421 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 135/164 2024-05-11T12:11:14.221 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T12:11:14.273 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T12:11:14.320 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T12:11:14.352 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T12:11:14.379 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T12:11:14.399 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-11T12:11:14.431 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:11:14.456 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 137/164 2024-05-11T12:11:14.456 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:14.456 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-11T12:11:14.456 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:11:14.456 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-11T12:11:14.456 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:14.499 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:11:14.523 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3601.g30850 138/164 2024-05-11T12:11:14.523 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:14.523 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-11T12:11:14.523 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:14.574 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:11:14.577 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T12:11:14.598 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 139/164 2024-05-11T12:11:14.599 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:14.599 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-11T12:11:14.599 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:11:14.599 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-11T12:11:14.599 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:14.616 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T12:11:14.646 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T12:11:14.676 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T12:11:14.722 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T12:11:14.917 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:11:14.941 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 140/164 2024-05-11T12:11:14.942 INFO:teuthology.orchestra.run.smithi064.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-11T12:11:14.942 INFO:teuthology.orchestra.run.smithi064.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-11T12:11:14.942 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:11:14.942 INFO:teuthology.orchestra.run.smithi064.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-11T12:11:14.942 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:15.327 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:15.369 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:15.390 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:16.306 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T12:11:16.399 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T12:11:16.452 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T12:11:16.569 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T12:11:16.602 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T12:11:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T12:11:16.628 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:11:16.721 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:11:16.753 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T12:11:16.753 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:11:16.816 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T12:11:16.846 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:11:16.857 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T12:11:16.871 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T12:11:16.876 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:16.876 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:16.898 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T12:11:16.930 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T12:11:16.957 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T12:11:17.066 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:17.066 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:11:17.149 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T12:11:17.149 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:11:17.227 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:17.237 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T12:11:17.267 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T12:11:17.297 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T12:11:17.343 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T12:11:18.011 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:18.050 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:18.071 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:18.990 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T12:11:19.080 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T12:11:19.123 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T12:11:19.215 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T12:11:19.250 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T12:11:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T12:11:19.285 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:11:19.361 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:11:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T12:11:19.394 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:11:19.459 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:11:19.484 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T12:11:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:19.490 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:19.689 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:19.689 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:11:19.790 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:11:19.829 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 141/164 2024-05-11T12:11:19.856 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:19.871 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 142/164 2024-05-11T12:11:19.912 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 143/164 2024-05-11T12:11:19.943 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-11T12:11:19.970 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-11T12:11:20.157 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 146/164 2024-05-11T12:11:20.200 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-11T12:11:20.230 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 148/164 2024-05-11T12:11:20.260 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-11T12:11:20.306 INFO:teuthology.orchestra.run.smithi064.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-11T12:11:20.944 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:20.978 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:20.999 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-11T12:11:21.914 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 152/164 2024-05-11T12:11:22.007 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-11T12:11:22.049 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-11T12:11:22.134 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-11T12:11:22.169 INFO:teuthology.orchestra.run.smithi064.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-11T12:11:22.195 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-11T12:11:22.195 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:11:22.262 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-11T12:11:22.302 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-11T12:11:22.302 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:11:22.388 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-11T12:11:22.413 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-11T12:11:22.419 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:22.419 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T12:11:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T12:11:22.575 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T12:11:22.576 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T12:11:22.577 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T12:11:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T12:11:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T12:11:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T12:11:22.590 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T12:11:22.591 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T12:11:22.592 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T12:11:22.593 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T12:11:22.594 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T12:11:22.595 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T12:11:22.596 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T12:11:22.597 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T12:11:22.598 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T12:11:22.599 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T12:11:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T12:11:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T12:11:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T12:11:22.600 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T12:11:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-11T12:11:22.615 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:11:22.691 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-11T12:11:22.758 INFO:teuthology.orchestra.run.smithi064.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T12:11:25.417 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T12:11:25.418 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T12:11:25.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T12:11:25.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T12:11:25.419 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T12:11:25.433 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T12:11:25.433 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T12:11:25.433 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T12:11:25.433 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T12:11:25.433 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T12:11:25.433 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T12:11:25.434 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T12:11:25.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T12:11:25.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T12:11:25.437 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T12:11:25.438 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T12:11:25.439 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T12:11:25.440 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T12:11:25.441 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T12:11:25.442 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T12:11:25.443 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T12:11:25.444 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T12:11:25.445 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T12:11:25.466 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:25.466 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:25.466 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T12:11:25.467 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.468 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T12:11:25.469 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.470 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T12:11:25.471 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T12:11:25.472 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T12:11:25.473 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T12:11:25.474 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T12:11:25.475 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T12:11:25.476 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T12:11:25.477 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:25.478 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T12:11:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T12:11:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T12:11:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T12:11:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T12:11:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T12:11:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T12:11:25.479 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:11:25.480 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-05-11T12:11:25.868 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-2:19.0.0-3601.g30850680.el9.x86_64 1/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 3/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 4/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3601.g30850 5/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 6/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 7/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 8/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 9/164 2024-05-11T12:11:28.267 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 10/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 11/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 12/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 13/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 14/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_ 15/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 16/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 17/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados-devel-2:19.0.0-3601.g30850680.el9.x86_6 18/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libradosstriper1-2:19.0.0-3601.g30850680.el9.x86 19/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librgw2-2:19.0.0-3601.g30850680.el9.x86_64 20/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3601.g30850680.el 21/164 2024-05-11T12:11:28.268 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ceph-common-2:19.0.0-3601.g30850680.el9. 22/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_6 23/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 24/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 25/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 26/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 27/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 28/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 29/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3601.g30850680. 30/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noa 31/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.n 32/164 2024-05-11T12:11:28.269 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3601.g308 33/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el 34/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 35/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3601.g30850680.e 36/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 37/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephadm-2:19.0.0-3601.g30850680.el9.noarch 38/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 39/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-11T12:11:28.270 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-11T12:11:28.271 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-11T12:11:28.272 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-11T12:11:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-11T12:11:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-11T12:11:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-11T12:11:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-11T12:11:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-11T12:11:28.273 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-11T12:11:28.274 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-11T12:11:28.275 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-11T12:11:28.276 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-11T12:11:28.277 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-11T12:11:28.278 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-11T12:11:28.279 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:19.0.0-3601.g30850680.el9.x86_64 149/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:19.0.0-3601.g30850680.el9.x86_64 151/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-11T12:11:28.280 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-11T12:11:28.281 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:28.301 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.302 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.303 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T12:11:28.304 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T12:11:28.305 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T12:11:28.306 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T12:11:28.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T12:11:28.308 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T12:11:28.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T12:11:28.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T12:11:28.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T12:11:28.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T12:11:28.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T12:11:28.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T12:11:28.309 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T12:11:28.310 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T12:11:28.311 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T12:11:28.312 INFO:teuthology.orchestra.run.smithi050.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T12:11:28.313 INFO:teuthology.orchestra.run.smithi050.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T12:11:28.314 INFO:teuthology.orchestra.run.smithi050.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T12:11:28.314 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T12:11:28.314 INFO:teuthology.orchestra.run.smithi050.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T12:11:28.314 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-11T12:11:28.314 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-11T12:11:28.725 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:30.895 INFO:teuthology.orchestra.run.smithi064.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-11T12:11:30.895 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:30.895 INFO:teuthology.orchestra.run.smithi064.stdout:Upgraded: 2024-05-11T12:11:30.895 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:30.895 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:30.895 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: librados2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: librbd1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout:Installed: 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: automake-1.16.2-8.el9.noarch 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: bison-3.7.4-5.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-base-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.896 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-grafana-dashboards-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-immutable-object-cache-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mds-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-dashboard-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-modules-core-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mgr-rook-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-mon-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-osd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-prometheus-alerts-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.897 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-radosgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-selinux-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-test-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: ceph-volume-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: cephadm-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-shell-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: cephfs-top-2:19.0.0-3601.g30850680.el9.noarch 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: flex-2.6.4-9.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-11T12:11:30.898 INFO:teuthology.orchestra.run.smithi064.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libcephfs2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libcephsqlite-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-11T12:11:30.899 INFO:teuthology.orchestra.run.smithi064.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: librados-devel-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: libradosstriper1-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: librgw2-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: lua-5.4.4-4.el9.x86_64 2024-05-11T12:11:30.900 INFO:teuthology.orchestra.run.smithi064.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-13.14-1.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-11T12:11:30.901 INFO:teuthology.orchestra.run.smithi064.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-argparse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ceph-common-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cephfs-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-11T12:11:30.902 INFO:teuthology.orchestra.run.smithi064.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-11T12:11:30.903 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-11T12:11:30.904 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-ply-3.11-14.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rados-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.905 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rgw-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-11T12:11:30.906 INFO:teuthology.orchestra.run.smithi064.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-fuse-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-mirror-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: rbd-nbd-2:19.0.0-3601.g30850680.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: re2-1:20211101-3.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-11T12:11:30.907 INFO:teuthology.orchestra.run.smithi064.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-11T12:11:30.908 INFO:teuthology.orchestra.run.smithi064.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-11T12:11:30.908 INFO:teuthology.orchestra.run.smithi064.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-11T12:11:30.908 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:11:30.908 INFO:teuthology.orchestra.run.smithi064.stdout:Complete! 2024-05-11T12:11:31.289 DEBUG:teuthology.parallel:result is None 2024-05-11T12:11:31.290 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:11:31.435 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T12:11:31.462 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-3601.g30850680.el9 2024-05-11T12:11:31.463 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T12:11:31.463 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T12:11:31.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:11:31.608 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T12:11:31.634 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-3601.g30850680.el9 2024-05-11T12:11:31.635 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T12:11:31.635 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T12:11:31.636 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:11:31.773 DEBUG:teuthology.orchestra.run.smithi064:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-11T12:11:31.796 INFO:teuthology.orchestra.run.smithi064.stdout:19.0.0-3601.g30850680.el9 2024-05-11T12:11:31.796 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3601.g30850680.el9 2024-05-11T12:11:31.796 INFO:teuthology.task.install:The correct ceph version 19.0.0-3601.g30850680 is installed. 2024-05-11T12:11:31.798 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-11T12:11:31.798 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:11:31.798 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T12:11:31.831 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:11:31.832 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T12:11:31.864 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:11:31.864 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-11T12:11:31.893 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-11T12:11:31.893 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:11:31.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T12:11:31.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T12:11:31.994 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:11:31.994 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T12:11:32.023 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T12:11:32.091 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:11:32.091 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/daemon-helper 2024-05-11T12:11:32.119 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-11T12:11:32.185 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-11T12:11:32.185 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:11:32.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T12:11:32.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T12:11:32.285 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:11:32.285 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T12:11:32.314 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T12:11:32.383 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:11:32.383 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-11T12:11:32.411 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-11T12:11:32.478 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-11T12:11:32.478 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:11:32.478 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T12:11:32.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T12:11:32.580 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:11:32.580 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T12:11:32.609 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T12:11:32.677 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:11:32.678 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/usr/bin/stdin-killer 2024-05-11T12:11:32.705 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-11T12:11:32.773 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-11T12:11:32.872 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'subvols': {'create': 2}} 2024-05-11T12:11:32.872 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:11:32.872 INFO:tasks.cephadm:Cluster fsid is 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:11:32.872 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-11T12:11:32.873 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.47', 'mon.b': '172.21.15.50', 'mon.c': '172.21.15.64'} 2024-05-11T12:11:32.873 INFO:tasks.cephadm:First mon is mon.a on smithi047 2024-05-11T12:11:32.873 INFO:tasks.cephadm:First mgr is x 2024-05-11T12:11:32.873 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-11T12:11:32.873 DEBUG:teuthology.orchestra.run.smithi047:> sudo hostname $(hostname -s) 2024-05-11T12:11:32.904 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-05-11T12:11:32.934 DEBUG:teuthology.orchestra.run.smithi064:> sudo hostname $(hostname -s) 2024-05-11T12:11:32.964 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-11T12:11:32.964 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:11:33.189 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '308506802b0bdea4df7050cfc1fcbdc04fea6a6e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79287/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3601-g30850680', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3601.g30850680'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-10 12:10:01.412462', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240510.053205', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-11T12:11:33.407 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-testing-20240510.053205, sha1 308506802b0bdea4df7050cfc1fcbdc04fea6a6e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:11:33.408 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm 2024-05-11T12:11:33.408 DEBUG:teuthology.orchestra.run.smithi047:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:34.017 INFO:teuthology.orchestra.run.smithi047.stdout:-rw-r--r--. 1 ubuntu ubuntu 782934 May 11 12:11 /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:34.017 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:34.624 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 782934 May 11 12:11 /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:34.624 DEBUG:teuthology.orchestra.run.smithi064:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240510.053205/308506802b0bdea4df7050cfc1fcbdc04fea6a6e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:35.081 INFO:teuthology.orchestra.run.smithi064.stdout:-rw-r--r--. 1 ubuntu ubuntu 782934 May 11 12:11 /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:35.082 DEBUG:teuthology.orchestra.run.smithi047:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:35.102 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:35.121 DEBUG:teuthology.orchestra.run.smithi064:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-11T12:11:35.156 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e on all hosts... 2024-05-11T12:11:35.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e pull 2024-05-11T12:11:35.159 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e pull 2024-05-11T12:11:35.164 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e pull 2024-05-11T12:11:36.646 INFO:teuthology.orchestra.run.smithi064.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T12:11:36.677 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T12:11:36.771 INFO:teuthology.orchestra.run.smithi047.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T12:12:11.889 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:12:11.890 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T12:12:11.890 INFO:teuthology.orchestra.run.smithi047.stdout: "image_id": "3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305", 2024-05-11T12:12:11.890 INFO:teuthology.orchestra.run.smithi047.stdout: "repo_digests": [ 2024-05-11T12:12:11.890 INFO:teuthology.orchestra.run.smithi047.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8" 2024-05-11T12:12:11.891 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:12:11.891 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:12:25.930 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-05-11T12:12:25.930 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T12:12:25.930 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305", 2024-05-11T12:12:25.930 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-05-11T12:12:25.930 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8" 2024-05-11T12:12:25.931 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-05-11T12:12:25.931 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-05-11T12:12:27.171 INFO:teuthology.orchestra.run.smithi064.stdout:{ 2024-05-11T12:12:27.171 INFO:teuthology.orchestra.run.smithi064.stdout: "ceph_version": "ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev)", 2024-05-11T12:12:27.171 INFO:teuthology.orchestra.run.smithi064.stdout: "image_id": "3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305", 2024-05-11T12:12:27.171 INFO:teuthology.orchestra.run.smithi064.stdout: "repo_digests": [ 2024-05-11T12:12:27.172 INFO:teuthology.orchestra.run.smithi064.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8" 2024-05-11T12:12:27.172 INFO:teuthology.orchestra.run.smithi064.stdout: ] 2024-05-11T12:12:27.172 INFO:teuthology.orchestra.run.smithi064.stdout:} 2024-05-11T12:12:27.199 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph 2024-05-11T12:12:27.233 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-05-11T12:12:27.266 DEBUG:teuthology.orchestra.run.smithi064:> sudo mkdir -p /etc/ceph 2024-05-11T12:12:27.295 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /etc/ceph 2024-05-11T12:12:27.327 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-05-11T12:12:27.363 DEBUG:teuthology.orchestra.run.smithi064:> sudo chmod 777 /etc/ceph 2024-05-11T12:12:27.390 INFO:tasks.cephadm:Writing seed config... 2024-05-11T12:12:27.391 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-11T12:12:27.392 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-11T12:12:27.393 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-11T12:12:27.394 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-11T12:12:27.395 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-11T12:12:27.395 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-11T12:12:27.395 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-11T12:12:27.395 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:12:27.396 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-11T12:12:27.416 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9b1a9a48-0f8f-11ef-bc98-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-11T12:12:27.416 DEBUG:teuthology.orchestra.run.smithi047:mon.a> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a.service 2024-05-11T12:12:27.460 DEBUG:teuthology.orchestra.run.smithi047:mgr.x> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.x.service 2024-05-11T12:12:27.503 INFO:tasks.cephadm:Bootstrapping... 2024-05-11T12:12:27.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e -v bootstrap --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.47 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:12:27.708 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-05-11T12:12:27.708 INFO:teuthology.orchestra.run.smithi047.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e', '-v', 'bootstrap', '--fsid', '9b1a9a48-0f8f-11ef-bc98-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.47', '--skip-admin-label'] 2024-05-11T12:12:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-11T12:12:27.709 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying podman|docker is present... 2024-05-11T12:12:27.742 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-11T12:12:27.742 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying lvm2 is present... 2024-05-11T12:12:27.742 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying time synchronization is in place... 2024-05-11T12:12:27.750 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-11T12:12:27.750 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-11T12:12:27.756 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-11T12:12:27.756 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-05-11T12:12:27.763 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-05-11T12:12:27.770 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-05-11T12:12:27.770 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-05-11T12:12:27.770 INFO:teuthology.orchestra.run.smithi047.stdout:Repeating the final host check... 2024-05-11T12:12:27.795 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-11T12:12:27.795 INFO:teuthology.orchestra.run.smithi047.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-11T12:12:27.796 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl is present 2024-05-11T12:12:27.796 INFO:teuthology.orchestra.run.smithi047.stdout:lvcreate is present 2024-05-11T12:12:27.802 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-11T12:12:27.803 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-11T12:12:27.809 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-11T12:12:27.809 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-05-11T12:12:27.816 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout enabled 2024-05-11T12:12:27.822 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout active 2024-05-11T12:12:27.822 INFO:teuthology.orchestra.run.smithi047.stdout:Unit chronyd.service is enabled and running 2024-05-11T12:12:27.822 INFO:teuthology.orchestra.run.smithi047.stdout:Host looks OK 2024-05-11T12:12:27.823 INFO:teuthology.orchestra.run.smithi047.stdout:Cluster fsid: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:27.823 INFO:teuthology.orchestra.run.smithi047.stdout:Acquiring lock 139705304153824 on /run/cephadm/9b1a9a48-0f8f-11ef-bc98-c7b262605968.lock 2024-05-11T12:12:27.823 INFO:teuthology.orchestra.run.smithi047.stdout:Lock 139705304153824 acquired on /run/cephadm/9b1a9a48-0f8f-11ef-bc98-c7b262605968.lock 2024-05-11T12:12:27.823 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 3300 ... 2024-05-11T12:12:27.823 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying IP 172.21.15.47 port 6789 ... 2024-05-11T12:12:27.823 INFO:teuthology.orchestra.run.smithi047.stdout:Base mon IP(s) is [172.21.15.47:3300, 172.21.15.47:6789], mon addrv is [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-05-11T12:12:27.827 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.47 metric 100 2024-05-11T12:12:27.827 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.47 metric 100 2024-05-11T12:12:27.830 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-11T12:12:27.830 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-11T12:12:27.830 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-11T12:12:27.832 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-11T12:12:27.832 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-11T12:12:27.832 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-11T12:12:27.832 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-11T12:12:27.832 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-05-11T12:12:27.832 INFO:teuthology.orchestra.run.smithi047.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-11T12:12:27.833 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-05-11T12:12:27.833 INFO:teuthology.orchestra.run.smithi047.stdout:Mon IP `172.21.15.47` is in CIDR network `172.21.0.0/20` 2024-05-11T12:12:27.833 INFO:teuthology.orchestra.run.smithi047.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-11T12:12:27.834 INFO:teuthology.orchestra.run.smithi047.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-11T12:12:27.835 INFO:teuthology.orchestra.run.smithi047.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T12:12:28.239 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stdout 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 2024-05-11T12:12:28.240 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e... 2024-05-11T12:12:28.240 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Getting image source signatures 2024-05-11T12:12:28.240 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:f023f0e3722bad1958426d3c75a01dbae989f69dabcafcf09827f24f8b5e0d6b 2024-05-11T12:12:28.240 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-11T12:12:28.240 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Copying config sha256:3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 2024-05-11T12:12:28.240 INFO:teuthology.orchestra.run.smithi047.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-11T12:12:29.088 INFO:teuthology.orchestra.run.smithi047.stdout:ceph: stdout ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev) 2024-05-11T12:12:29.088 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph version: ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev) 2024-05-11T12:12:29.088 INFO:teuthology.orchestra.run.smithi047.stdout:Extracting ceph user uid/gid from container image... 2024-05-11T12:12:30.020 INFO:teuthology.orchestra.run.smithi047.stdout:stat: stdout 167 167 2024-05-11T12:12:30.020 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial keys... 2024-05-11T12:12:31.045 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQCuYD9m/2FWFBAAb9l0OBSRwL30aJO+uQcXIQ== 2024-05-11T12:12:31.951 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQCvYD9mlS1KFhAAaO58Xc4D3qJnx847TIKK7g== 2024-05-11T12:12:32.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph-authtool: stdout AQCwYD9mEnQcDxAAdsg7HYpDiAwgkSC57nNfWw== 2024-05-11T12:12:32.824 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial monmap... 2024-05-11T12:12:34.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-11T12:12:34.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-11T12:12:34.713 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-11T12:12:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool for a [v2:172.21.15.47:3300,v1:172.21.15.47:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-11T12:12:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-05-11T12:12:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: set fsid to 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-11T12:12:34.714 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:12:34.714 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mon... 2024-05-11T12:12:43.818 INFO:teuthology.orchestra.run.smithi047.stdout:create mon.a on 2024-05-11T12:12:44.305 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-11T12:12:44.529 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-11T12:12:44.737 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968.target. 2024-05-11T12:12:44.737 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968.target → /etc/systemd/system/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968.target. 2024-05-11T12:12:44.973 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a 2024-05-11T12:12:44.973 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a.service: Unit ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a.service not loaded. 2024-05-11T12:12:45.151 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968.target.wants/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@.service. 2024-05-11T12:12:45.710 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T12:12:45.710 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-05-11T12:12:45.718 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T12:12:45.718 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-05-11T12:12:45.718 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-05-11T12:12:45.718 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-05-11T12:12:45.718 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon to start... 2024-05-11T12:12:45.718 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mon... 2024-05-11T12:12:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout cluster: 2024-05-11T12:12:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout id: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-11T12:12:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:46.770 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout services: 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.577178s) 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout data: 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout pgs: 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.151+0000 7fd1650f2700 1 Processor -- start 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.151+0000 7fd1650f2700 1 -- start start 2024-05-11T12:12:46.771 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.151+0000 7fd1650f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd160107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.151+0000 7fd1650f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd16007cda0 con 0x7fd1601076c0 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd160107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd160107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56602/0 (socket says 172.21.15.47:56602) 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15ed9d700 1 -- 172.21.15.47:0/4201580690 learned_addr learned my addr 172.21.15.47:0/4201580690 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15ed9d700 1 -- 172.21.15.47:0/4201580690 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd160108020 con 0x7fd1601076c0 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15ed9d700 1 --2- 172.21.15.47:0/4201580690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd160107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd1500098a0 tx=0x7fd150009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c60197b60d3fc25 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15dd9b700 1 -- 172.21.15.47:0/4201580690 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd150004160 con 0x7fd1601076c0 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15dd9b700 1 -- 172.21.15.47:0/4201580690 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd150004300 con 0x7fd1601076c0 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.152+0000 7fd15dd9b700 1 -- 172.21.15.47:0/4201580690 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1500036a0 con 0x7fd1601076c0 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.153+0000 7fd1650f2700 1 -- 172.21.15.47:0/4201580690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 msgr2=0x7fd160107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.153+0000 7fd1650f2700 1 --2- 172.21.15.47:0/4201580690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd160107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd1500098a0 tx=0x7fd150009bb0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.153+0000 7fd1650f2700 1 -- 172.21.15.47:0/4201580690 shutdown_connections 2024-05-11T12:12:46.772 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.153+0000 7fd1650f2700 1 --2- 172.21.15.47:0/4201580690 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd160107ae0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.153+0000 7fd1650f2700 1 -- 172.21.15.47:0/4201580690 >> 172.21.15.47:0/4201580690 conn(0x7fd160103240 msgr2=0x7fd1601056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.153+0000 7fd1650f2700 1 -- 172.21.15.47:0/4201580690 shutdown_connections 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.153+0000 7fd1650f2700 1 -- 172.21.15.47:0/4201580690 wait complete. 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.154+0000 7fd1650f2700 1 Processor -- start 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.154+0000 7fd1650f2700 1 -- start start 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.154+0000 7fd1650f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd16019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.154+0000 7fd1650f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd160108550 con 0x7fd1601076c0 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.154+0000 7fd15ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd16019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.154+0000 7fd15ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd16019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56606/0 (socket says 172.21.15.47:56606) 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.154+0000 7fd15ed9d700 1 -- 172.21.15.47:0/3664987435 learned_addr learned my addr 172.21.15.47:0/3664987435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd15ed9d700 1 -- 172.21.15.47:0/3664987435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16019ecc0 con 0x7fd1601076c0 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd15ed9d700 1 --2- 172.21.15.47:0/3664987435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd16019e780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd150009fb0 tx=0x7fd150004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd147fff700 1 -- 172.21.15.47:0/3664987435 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd150003de0 con 0x7fd1601076c0 2024-05-11T12:12:46.773 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd147fff700 1 -- 172.21.15.47:0/3664987435 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd150011630 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16019ef50 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd147fff700 1 -- 172.21.15.47:0/3664987435 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd150011930 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1601a1bf0 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd147fff700 1 -- 172.21.15.47:0/3664987435 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd150024480 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.155+0000 7fd147fff700 1 -- 172.21.15.47:0/3664987435 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd15001a7e0 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.156+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd160064bb0 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.159+0000 7fd147fff700 1 -- 172.21.15.47:0/3664987435 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd15001aa80 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.232+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd1601a20d0 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.232+0000 7fd147fff700 1 -- 172.21.15.47:0/3664987435 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd15001ac60 con 0x7fd1601076c0 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.233+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 msgr2=0x7fd16019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.233+0000 7fd1650f2700 1 --2- 172.21.15.47:0/3664987435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd16019e780 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd150009fb0 tx=0x7fd150004750 comp rx=0 tx=0).stop 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.233+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 shutdown_connections 2024-05-11T12:12:46.774 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.233+0000 7fd1650f2700 1 --2- 172.21.15.47:0/3664987435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1601076c0 0x7fd16019e780 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:46.775 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.233+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 >> 172.21.15.47:0/3664987435 conn(0x7fd160103240 msgr2=0x7fd160194da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:46.775 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.233+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 shutdown_connections 2024-05-11T12:12:46.775 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:46.233+0000 7fd1650f2700 1 -- 172.21.15.47:0/3664987435 wait complete. 2024-05-11T12:12:46.775 INFO:teuthology.orchestra.run.smithi047.stdout:mon is available 2024-05-11T12:12:46.775 INFO:teuthology.orchestra.run.smithi047.stdout:Assimilating anything we can from ceph.conf... 2024-05-11T12:12:47.901 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:47 smithi047 ceph-mon[30006]: from='client.? 172.21.15.47:0/3122218901' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.47:3300,v1:172.21.15.47:6789] 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-11T12:12:47.911 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.211+0000 7fa5402f2700 1 Processor -- start 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.211+0000 7fa5402f2700 1 -- start start 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.211+0000 7fa5402f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa5381091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.211+0000 7fa5402f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381097c0 con 0x7fa538108dd0 2024-05-11T12:12:47.912 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53e08e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa5381091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53e08e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa5381091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56090/0 (socket says 172.21.15.47:56090) 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53e08e700 1 -- 172.21.15.47:0/2096784557 learned_addr learned my addr 172.21.15.47:0/2096784557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53e08e700 1 -- 172.21.15.47:0/2096784557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa538109fd0 con 0x7fa538108dd0 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53e08e700 1 --2- 172.21.15.47:0/2096784557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa5381091f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa528009e80 tx=0x7fa52800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46553a0d7158ff25 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53d08c700 1 -- 172.21.15.47:0/2096784557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa528017070 con 0x7fa538108dd0 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53d08c700 1 -- 172.21.15.47:0/2096784557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa528003a20 con 0x7fa538108dd0 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.212+0000 7fa53d08c700 1 -- 172.21.15.47:0/2096784557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa528003d20 con 0x7fa538108dd0 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.213+0000 7fa5402f2700 1 -- 172.21.15.47:0/2096784557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 msgr2=0x7fa5381091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.213+0000 7fa5402f2700 1 --2- 172.21.15.47:0/2096784557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa5381091f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa528009e80 tx=0x7fa52800b330 comp rx=0 tx=0).stop 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.213+0000 7fa5402f2700 1 -- 172.21.15.47:0/2096784557 shutdown_connections 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.213+0000 7fa5402f2700 1 --2- 172.21.15.47:0/2096784557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa5381091f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.213+0000 7fa5402f2700 1 -- 172.21.15.47:0/2096784557 >> 172.21.15.47:0/2096784557 conn(0x7fa5381043b0 msgr2=0x7fa5381067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:47.913 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.213+0000 7fa5402f2700 1 -- 172.21.15.47:0/2096784557 shutdown_connections 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.213+0000 7fa5402f2700 1 -- 172.21.15.47:0/2096784557 wait complete. 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa5402f2700 1 Processor -- start 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa5402f2700 1 -- start start 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa5402f2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa53807f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa5402f2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa53810a500 con 0x7fa538108dd0 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa53e08e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa53807f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa53e08e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa53807f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56104/0 (socket says 172.21.15.47:56104) 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa53e08e700 1 -- 172.21.15.47:0/3122218901 learned_addr learned my addr 172.21.15.47:0/3122218901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa53e08e700 1 -- 172.21.15.47:0/3122218901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53807f8d0 con 0x7fa538108dd0 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.214+0000 7fa53e08e700 1 --2- 172.21.15.47:0/3122218901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa53807f390 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa528004000 tx=0x7fa528004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.215+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa528022020 con 0x7fa538108dd0 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.215+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa528018040 con 0x7fa538108dd0 2024-05-11T12:12:47.914 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.215+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53807dac0 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.215+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa528017070 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.215+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa53807e000 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.215+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa528025020 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.215+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa52800f050 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.216+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa538064bb0 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.218+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa52802c030 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.289+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa538195dd0 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.292+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fa528012610 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.293+0000 7fa5377fe700 1 -- 172.21.15.47:0/3122218901 <== mon.0 v2:172.21.15.47:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa528035020 con 0x7fa538108dd0 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.294+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 msgr2=0x7fa53807f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.294+0000 7fa5402f2700 1 --2- 172.21.15.47:0/3122218901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa53807f390 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa528004000 tx=0x7fa528004ad0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.294+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 shutdown_connections 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.294+0000 7fa5402f2700 1 --2- 172.21.15.47:0/3122218901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa538108dd0 0x7fa53807f390 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:47.915 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.294+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 >> 172.21.15.47:0/3122218901 conn(0x7fa5381043b0 msgr2=0x7fa5381931d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:47.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.294+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 shutdown_connections 2024-05-11T12:12:47.916 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:47.294+0000 7fa5402f2700 1 -- 172.21.15.47:0/3122218901 wait complete. 2024-05-11T12:12:47.916 INFO:teuthology.orchestra.run.smithi047.stdout:Generating new minimal ceph.conf... 2024-05-11T12:12:49.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:48 smithi047 ceph-mon[30006]: from='client.? 172.21.15.47:0/3183516122' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:12:49.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.353+0000 7feec0d3e700 1 Processor -- start 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.353+0000 7feec0d3e700 1 -- start start 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feec0d3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feec0d3e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feebc107aa0 con 0x7feebc1070b0 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeba59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeba59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56120/0 (socket says 172.21.15.47:56120) 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeba59c700 1 -- 172.21.15.47:0/3459441829 learned_addr learned my addr 172.21.15.47:0/3459441829 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeba59c700 1 -- 172.21.15.47:0/3459441829 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc10a040 con 0x7feebc1070b0 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeba59c700 1 --2- 172.21.15.47:0/3459441829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1074d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feeac009e80 tx=0x7feeac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886b17c53c21056b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeb959a700 1 -- 172.21.15.47:0/3459441829 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac017070 con 0x7feebc1070b0 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeb959a700 1 -- 172.21.15.47:0/3459441829 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feeac00b8c0 con 0x7feebc1070b0 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.354+0000 7feeb959a700 1 -- 172.21.15.47:0/3459441829 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac004490 con 0x7feebc1070b0 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.355+0000 7feec0d3e700 1 -- 172.21.15.47:0/3459441829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 msgr2=0x7feebc1074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.355+0000 7feec0d3e700 1 --2- 172.21.15.47:0/3459441829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1074d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feeac009e80 tx=0x7feeac00b330 comp rx=0 tx=0).stop 2024-05-11T12:12:49.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.355+0000 7feec0d3e700 1 -- 172.21.15.47:0/3459441829 shutdown_connections 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.355+0000 7feec0d3e700 1 --2- 172.21.15.47:0/3459441829 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1074d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.355+0000 7feec0d3e700 1 -- 172.21.15.47:0/3459441829 >> 172.21.15.47:0/3459441829 conn(0x7feebc1029c0 msgr2=0x7feebc104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.355+0000 7feec0d3e700 1 -- 172.21.15.47:0/3459441829 shutdown_connections 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.355+0000 7feec0d3e700 1 -- 172.21.15.47:0/3459441829 wait complete. 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.356+0000 7feec0d3e700 1 Processor -- start 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.356+0000 7feec0d3e700 1 -- start start 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.356+0000 7feec0d3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.356+0000 7feec0d3e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feebc10d3f0 con 0x7feebc1070b0 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.356+0000 7feeba59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.356+0000 7feeba59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1098c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56130/0 (socket says 172.21.15.47:56130) 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.356+0000 7feeba59c700 1 -- 172.21.15.47:0/3183516122 learned_addr learned my addr 172.21.15.47:0/3183516122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feeba59c700 1 -- 172.21.15.47:0/3183516122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc109e00 con 0x7feebc1070b0 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feeba59c700 1 --2- 172.21.15.47:0/3183516122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1098c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feeac00b7b0 tx=0x7feeac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feeab7fe700 1 -- 172.21.15.47:0/3183516122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac020020 con 0x7feebc1070b0 2024-05-11T12:12:49.242 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feeab7fe700 1 -- 172.21.15.47:0/3183516122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feeac012430 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feeab7fe700 1 -- 172.21.15.47:0/3183516122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeac017040 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc107ff0 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc108530 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feeab7fe700 1 -- 172.21.15.47:0/3183516122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feeac023020 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.357+0000 7feeab7fe700 1 -- 172.21.15.47:0/3183516122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feeac00f050 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.358+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feebc064bb0 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.361+0000 7feeab7fe700 1 -- 172.21.15.47:0/3183516122 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7feeac02a020 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.430+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7feebc108870 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.431+0000 7feeab7fe700 1 -- 172.21.15.47:0/3183516122 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7feeac00b860 con 0x7feebc1070b0 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.431+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 msgr2=0x7feebc1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.431+0000 7feec0d3e700 1 --2- 172.21.15.47:0/3183516122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1098c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feeac00b7b0 tx=0x7feeac00be20 comp rx=0 tx=0).stop 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.431+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 shutdown_connections 2024-05-11T12:12:49.243 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.431+0000 7feec0d3e700 1 --2- 172.21.15.47:0/3183516122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feebc1070b0 0x7feebc1098c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:49.244 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.431+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 >> 172.21.15.47:0/3183516122 conn(0x7feebc1029c0 msgr2=0x7feebc103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:49.244 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.432+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 shutdown_connections 2024-05-11T12:12:49.244 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:48.432+0000 7feec0d3e700 1 -- 172.21.15.47:0/3183516122 wait complete. 2024-05-11T12:12:49.244 INFO:teuthology.orchestra.run.smithi047.stdout:Restarting the monitor... 2024-05-11T12:12:49.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:49 smithi047 systemd[1]: Stopping Ceph mon.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:12:49.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:49 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a[30002]: 2024-05-11T12:12:49.687+0000 7f6e0bc11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:12:49.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:49 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a[30002]: 2024-05-11T12:12:49.687+0000 7f6e0bc11700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-11T12:12:50.245 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:49 smithi047 podman[30249]: 2024-05-11 12:12:49.953982959 +0000 UTC m=+0.532671228 container died ac2e120e93be6d7f1d0ba9483e7a01bee16b753112c7db1813b2b033e302d06c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-11T12:12:50.519 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 podman[30249]: 2024-05-11 12:12:50.369949381 +0000 UTC m=+0.948637647 container cleanup ac2e120e93be6d7f1d0ba9483e7a01bee16b753112c7db1813b2b033e302d06c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507) 2024-05-11T12:12:50.520 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 bash[30249]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a 2024-05-11T12:12:50.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 podman[30263]: 2024-05-11 12:12:50.520861994 +0000 UTC m=+0.569722989 container remove ac2e120e93be6d7f1d0ba9483e7a01bee16b753112c7db1813b2b033e302d06c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux ) 2024-05-11T12:12:50.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 systemd[1]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-11T12:12:50.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 systemd[1]: Stopped Ceph mon.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:12:50.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 systemd[1]: Starting Ceph mon.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:12:51.068 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 podman[30357]: 2024-05-11 12:12:50.872193339 +0000 UTC m=+0.018468286 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:12:51.068 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:50 smithi047 podman[30357]: 2024-05-11 12:12:50.980296321 +0000 UTC m=+0.126571258 container create da3e44580bab25d4be199df2110939078700331ba83f2d22b51ea7a75c5c28c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:12:51.236 INFO:teuthology.orchestra.run.smithi047.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-11T12:12:51.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 podman[30357]: 2024-05-11 12:12:51.149243182 +0000 UTC m=+0.295518120 container init da3e44580bab25d4be199df2110939078700331ba83f2d22b51ea7a75c5c28c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:12:51.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 podman[30357]: 2024-05-11 12:12:51.152340947 +0000 UTC m=+0.298615885 container start da3e44580bab25d4be199df2110939078700331ba83f2d22b51ea7a75c5c28c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-11T12:12:51.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: set uid:gid to 167:167 (ceph:ceph) 2024-05-11T12:12:51.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: ceph version 19.0.0-3601-g30850680 (308506802b0bdea4df7050cfc1fcbdc04fea6a6e) squid (dev), process ceph-mon, pid 2 2024-05-11T12:12:51.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: pidfile_write: ignore empty --pid-file 2024-05-11T12:12:51.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: load: jerasure load: lrc 2024-05-11T12:12:51.361 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: RocksDB version: 7.9.2 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Git sha 0 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Compile date 2024-05-10 10:41:35 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: DB SUMMARY 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: DB Session ID: K2AJEEKU6G5WE9RTTUCA 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: CURRENT file: CURRENT 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: IDENTITY file: IDENTITY 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104066 ; 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.error_if_exists: 0 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.create_if_missing: 0 2024-05-11T12:12:51.362 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.paranoid_checks: 1 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.env: 0x559ac81a3cc0 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.fs: PosixFileSystem 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.info_log: 0x559ac959ddc0 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_file_opening_threads: 16 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.statistics: (nil) 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.use_fsync: 0 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_log_file_size: 0 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.keep_log_file_num: 1000 2024-05-11T12:12:51.363 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.recycle_log_file_num: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.allow_fallocate: 1 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.allow_mmap_reads: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.allow_mmap_writes: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.use_direct_reads: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.create_missing_column_families: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.db_log_dir: 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.wal_dir: 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-11T12:12:51.364 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.advise_random_on_open: 1 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.db_write_buffer_size: 0 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.write_buffer_manager: 0x559ac956f0e0 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.rate_limiter: (nil) 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.wal_recovery_mode: 2 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.enable_thread_tracking: 0 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.enable_pipelined_write: 0 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.unordered_write: 0 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-11T12:12:51.365 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-11T12:12:51.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-11T12:12:51.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.row_cache: None 2024-05-11T12:12:51.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.wal_filter: None 2024-05-11T12:12:51.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-11T12:12:51.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.allow_ingest_behind: 0 2024-05-11T12:12:51.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.two_write_queues: 0 2024-05-11T12:12:51.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.manual_wal_flush: 0 2024-05-11T12:12:51.367 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.wal_compression: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.atomic_flush: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.log_readahead_size: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.best_efforts_recovery: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.allow_data_in_errors: 0 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.db_host_id: __hostname__ 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_background_jobs: 2 2024-05-11T12:12:51.368 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_background_compactions: -1 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_subcompactions: 1 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_total_wal_size: 0 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_open_files: -1 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bytes_per_sync: 0 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-11T12:12:51.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_readahead_size: 0 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_background_flushes: -1 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Compression algorithms supported: 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kZSTD supported: 0 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kXpressCompression supported: 0 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kLZ4HCCompression supported: 1 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kZlibCompression supported: 1 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kSnappyCompression supported: 1 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kLZ4Compression supported: 1 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: kBZip2Compression supported: 0 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-11T12:12:51.370 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.merge_operator: 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_filter: None 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_filter_factory: None 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.sst_partitioner_factory: None 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559ac959df80) 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks: 1 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout: pin_top_level_index_and_filter: 1 2024-05-11T12:12:51.371 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_type: 0 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_index_type: 0 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_shortening: 1 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: checksum: 4 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: no_block_cache: 0 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache: 0x559ac95a11f0 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_name: BinnedLRUCache 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_options: 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: capacity : 536870912 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_shard_bits : 4 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: strict_capacity_limit : 0 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: high_pri_pool_ratio: 0.000 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_cache_compressed: (nil) 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: persistent_cache: (nil) 2024-05-11T12:12:51.372 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size: 4096 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_size_deviation: 10 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_restart_interval: 16 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: index_block_restart_interval: 1 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: metadata_block_size: 4096 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: partition_filters: 0 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: use_delta_encoding: 1 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: filter_policy: bloomfilter 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: whole_key_filtering: 1 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: verify_compression: 0 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: read_amp_bytes_per_bit: 0 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: format_version: 5 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_index_compression: 1 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: block_align: 0 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: max_auto_readahead_size: 262144 2024-05-11T12:12:51.373 INFO:journalctl@ceph.mon.a.smithi047.stdout: prepopulate_block_cache: 0 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout: initial_auto_readahead_size: 8192 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout: num_file_reads_for_auto_readahead: 2 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.write_buffer_size: 33554432 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_write_buffer_number: 2 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression: NoCompression 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression: Disabled 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.prefix_extractor: nullptr 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.num_levels: 7 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-11T12:12:51.374 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.level: 32767 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.strategy: 0 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-11T12:12:51.375 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.enabled: false 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.target_file_size_base: 67108864 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-11T12:12:51.376 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.arena_block_size: 1048576 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.disable_auto_compactions: 0 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-11T12:12:51.377 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.inplace_update_support: 0 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.bloom_locality: 0 2024-05-11T12:12:51.378 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.max_successive_merges: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.paranoid_file_checks: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.force_consistency_checks: 1 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.report_bg_io_stats: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.ttl: 2592000 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.enable_blob_files: false 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.min_blob_size: 0 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.blob_file_size: 268435456 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-11T12:12:51.379 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.blob_file_starting_level: 0 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 83486b29-f0a3-4584-be12-6d99315d4b4b 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715429571182140, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715429571183241, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99294, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 97240, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13917, "raw_average_key_size": 49, "raw_value_size": 89634, "raw_average_value_size": 318, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715429571, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "83486b29-f0a3-4584-be12-6d99315d4b4b", "db_session_id": "K2AJEEKU6G5WE9RTTUCA", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715429571183329, "job": 1, "event": "recovery_finished"} 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-11T12:12:51.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559ac96be000 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: DB pointer 0x559ac96aa000 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** DB Stats ** 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-05-11T12:12:51.381 INFO:journalctl@ceph.mon.a.smithi047.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: L0 2/0 98.83 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: Sum 2/0 98.83 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** Compaction Stats [default] ** 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-11T12:12:51.382 INFO:journalctl@ceph.mon.a.smithi047.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: Cumulative compaction: 0.00 GB write, 29.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: Interval compaction: 0.00 GB write, 29.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache BinnedLRUCache@0x559ac95a11f0#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: starting mon.a rank 0 at public addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] at bind addrs [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???) e1 preinit fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:51.383 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???).mds e1 new map 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???).mds e1 print_map 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout: e1 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout: legacy client fscid: -1 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout: 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout: No filesystems configured 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 ceph-mon[30375]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 bash[30357]: da3e44580bab25d4be199df2110939078700331ba83f2d22b51ea7a75c5c28c5 2024-05-11T12:12:51.384 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:51 smithi047 systemd[1]: Started Ceph mon.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:12:52.357 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.745+0000 7f89afddf700 1 Processor -- start 2024-05-11T12:12:52.362 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.745+0000 7f89afddf700 1 -- start start 2024-05-11T12:12:52.362 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.745+0000 7f89afddf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a807db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.745+0000 7f89afddf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f89a807e0d0 con 0x7f89a807f6a0 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89adb7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a807db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89adb7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a807db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56142/0 (socket says 172.21.15.47:56142) 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89adb7b700 1 -- 172.21.15.47:0/3422716933 learned_addr learned my addr 172.21.15.47:0/3422716933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89adb7b700 1 -- 172.21.15.47:0/3422716933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a807e250 con 0x7f89a807f6a0 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89adb7b700 1 --2- 172.21.15.47:0/3422716933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a807db00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f89a40098a0 tx=0x7f89a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc264d733cdea92 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89acb79700 1 -- 172.21.15.47:0/3422716933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89a4004160 con 0x7f89a807f6a0 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89acb79700 1 -- 172.21.15.47:0/3422716933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f89a4004300 con 0x7f89a807f6a0 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.746+0000 7f89acb79700 1 -- 172.21.15.47:0/3422716933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89a4011af0 con 0x7f89a807f6a0 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.747+0000 7f89afddf700 1 -- 172.21.15.47:0/3422716933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 msgr2=0x7f89a807db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.747+0000 7f89afddf700 1 --2- 172.21.15.47:0/3422716933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a807db00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f89a40098a0 tx=0x7f89a4009bb0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.747+0000 7f89afddf700 1 -- 172.21.15.47:0/3422716933 shutdown_connections 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.747+0000 7f89afddf700 1 --2- 172.21.15.47:0/3422716933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a807db00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.363 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.747+0000 7f89afddf700 1 -- 172.21.15.47:0/3422716933 >> 172.21.15.47:0/3422716933 conn(0x7f89a81043f0 msgr2=0x7f89a8106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.747+0000 7f89afddf700 1 -- 172.21.15.47:0/3422716933 shutdown_connections 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.747+0000 7f89afddf700 1 -- 172.21.15.47:0/3422716933 wait complete. 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89afddf700 1 Processor -- start 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89afddf700 1 -- start start 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89afddf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89afddf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f89a810f400 con 0x7f89a807f6a0 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89adb7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89adb7b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56158/0 (socket says 172.21.15.47:56158) 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89adb7b700 1 -- 172.21.15.47:0/1401498726 learned_addr learned my addr 172.21.15.47:0/1401498726 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89adb7b700 1 -- 172.21.15.47:0/1401498726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a819f450 con 0x7f89a807f6a0 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.748+0000 7f89adb7b700 1 --2- 172.21.15.47:0/1401498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a819ef10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f89a4011ac0 tx=0x7f89a4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.749+0000 7f899effd700 1 -- 172.21.15.47:0/1401498726 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89a4003b20 con 0x7f89a807f6a0 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.749+0000 7f899effd700 1 -- 172.21.15.47:0/1401498726 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f89a401a8c0 con 0x7f89a807f6a0 2024-05-11T12:12:52.364 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.749+0000 7f899effd700 1 -- 172.21.15.47:0/1401498726 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89a40248d0 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.749+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a819f6e0 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.749+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a81a23d0 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.749+0000 7f899effd700 1 -- 172.21.15.47:0/1401498726 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f89a4022020 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.749+0000 7f899effd700 1 -- 172.21.15.47:0/1401498726 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f89a40193c0 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.750+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a8064bb0 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.752+0000 7f899effd700 1 -- 172.21.15.47:0/1401498726 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f89a4042ec0 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.822+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f89a81a25a0 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.824+0000 7f899effd700 1 -- 172.21.15.47:0/1401498726 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f89a4031020 con 0x7f89a807f6a0 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.825+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 msgr2=0x7f89a819ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.825+0000 7f89afddf700 1 --2- 172.21.15.47:0/1401498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a819ef10 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f89a4011ac0 tx=0x7f89a4011ba0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.825+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 shutdown_connections 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.825+0000 7f89afddf700 1 --2- 172.21.15.47:0/1401498726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89a807f6a0 0x7f89a819ef10 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.825+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 >> 172.21.15.47:0/1401498726 conn(0x7f89a81043f0 msgr2=0x7f89a8104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:52.365 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.825+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 shutdown_connections 2024-05-11T12:12:52.366 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:51.825+0000 7f89afddf700 1 -- 172.21.15.47:0/1401498726 wait complete. 2024-05-11T12:12:52.366 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-11T12:12:52.380 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:12:52.380 INFO:teuthology.orchestra.run.smithi047.stdout:Creating mgr... 2024-05-11T12:12:52.380 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-11T12:12:52.380 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-11T12:12:52.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-11T12:12:52.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: monmap epoch 1 2024-05-11T12:12:52.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:12:52.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: last_changed 2024-05-11T12:12:33.150058+0000 2024-05-11T12:12:52.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: created 2024-05-11T12:12:33.150058+0000 2024-05-11T12:12:52.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: min_mon_release 19 (squid) 2024-05-11T12:12:52.446 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: election_strategy: 1 2024-05-11T12:12:52.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-05-11T12:12:52.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: fsmap 2024-05-11T12:12:52.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: osdmap e1: 0 total, 0 up, 0 in 2024-05-11T12:12:52.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: mgrmap e1: no daemons active 2024-05-11T12:12:52.447 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:52 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1401498726' entity='client.admin' 2024-05-11T12:12:52.616 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.x 2024-05-11T12:12:52.616 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.x.service: Unit ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.x.service not loaded. 2024-05-11T12:12:52.776 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968.target.wants/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@.service. 2024-05-11T12:12:53.366 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T12:12:53.366 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-05-11T12:12:53.374 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T12:12:53.374 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-05-11T12:12:53.374 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-05-11T12:12:53.374 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to enable service . firewalld.service is not available 2024-05-11T12:12:53.383 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T12:12:53.383 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-05-11T12:12:53.391 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T12:12:53.391 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-05-11T12:12:53.391 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-05-11T12:12:53.391 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-11T12:12:53.391 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr to start... 2024-05-11T12:12:53.391 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr... 2024-05-11T12:12:53.521 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:53.434+0000 7f9b16cfe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T12:12:53.774 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:53.630+0000 7f9b16cfe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:12:53.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:53.739+0000 7f9b16cfe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T12:12:54.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:54 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2343457538' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T12:12:54.178 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:54 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:54.089+0000 7f9b16cfe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T12:12:54.524 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:54 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:54.178+0000 7f9b16cfe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T12:12:54.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:54.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:12:54.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "9b1a9a48-0f8f-11ef-bc98-c7b262605968", 2024-05-11T12:12:54.638 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-11T12:12:54.639 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T12:12:54.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:54.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T12:12:54.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T12:12:54.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:54.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T12:12:54.640 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T12:12:54.641 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T12:12:54.642 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T12:12:45.658187+0000", 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.860+0000 7f2863314700 1 Processor -- start 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.860+0000 7f2863314700 1 -- start start 2024-05-11T12:12:54.643 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.860+0000 7f2863314700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1074c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.860+0000 7f2863314700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f285c107a90 con 0x7f285c1070a0 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.860+0000 7f28610b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.860+0000 7f28610b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56176/0 (socket says 172.21.15.47:56176) 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.860+0000 7f28610b0700 1 -- 172.21.15.47:0/2484782398 learned_addr learned my addr 172.21.15.47:0/2484782398 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.861+0000 7f28610b0700 1 -- 172.21.15.47:0/2484782398 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c107c10 con 0x7f285c1070a0 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.861+0000 7f28610b0700 1 --2- 172.21.15.47:0/2484782398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1074c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f28580098a0 tx=0x7f2858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf00d3e5e2e334c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.861+0000 7f2853fff700 1 -- 172.21.15.47:0/2484782398 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858004160 con 0x7f285c1070a0 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.861+0000 7f2853fff700 1 -- 172.21.15.47:0/2484782398 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2858004300 con 0x7f285c1070a0 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.861+0000 7f2853fff700 1 -- 172.21.15.47:0/2484782398 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858011af0 con 0x7f285c1070a0 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.861+0000 7f2863314700 1 -- 172.21.15.47:0/2484782398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 msgr2=0x7f285c1074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.861+0000 7f2863314700 1 --2- 172.21.15.47:0/2484782398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1074c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f28580098a0 tx=0x7f2858009bb0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.862+0000 7f2863314700 1 -- 172.21.15.47:0/2484782398 shutdown_connections 2024-05-11T12:12:54.644 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.862+0000 7f2863314700 1 --2- 172.21.15.47:0/2484782398 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1074c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.862+0000 7f2863314700 1 -- 172.21.15.47:0/2484782398 >> 172.21.15.47:0/2484782398 conn(0x7f285c1029c0 msgr2=0x7f285c104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.862+0000 7f2863314700 1 -- 172.21.15.47:0/2484782398 shutdown_connections 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.862+0000 7f2863314700 1 -- 172.21.15.47:0/2484782398 wait complete. 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.862+0000 7f2863314700 1 Processor -- start 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.862+0000 7f2863314700 1 -- start start 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f2863314700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1099a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f2863314700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f285c10af00 con 0x7f285c1070a0 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28610b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28610b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56182/0 (socket says 172.21.15.47:56182) 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28610b0700 1 -- 172.21.15.47:0/2343457538 learned_addr learned my addr 172.21.15.47:0/2343457538 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28610b0700 1 -- 172.21.15.47:0/2343457538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c109ee0 con 0x7f285c1070a0 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28610b0700 1 --2- 172.21.15.47:0/2343457538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1099a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f28580117b0 tx=0x7f2858011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28527fc700 1 -- 172.21.15.47:0/2343457538 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858003af0 con 0x7f285c1070a0 2024-05-11T12:12:54.645 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c1080d0 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28527fc700 1 -- 172.21.15.47:0/2343457538 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2858020070 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f28527fc700 1 -- 172.21.15.47:0/2343457538 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2858019630 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.863+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c108610 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.864+0000 7f28527fc700 1 -- 172.21.15.47:0/2343457538 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2858019c30 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.864+0000 7f28527fc700 1 -- 172.21.15.47:0/2343457538 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f285800ba20 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.865+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f285c10d350 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.869+0000 7f28527fc700 1 -- 172.21.15.47:0/2343457538 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f28580042a0 con 0x7f285c1070a0 2024-05-11T12:12:54.646 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.953+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f285c108a80 con 0x7f285c1070a0 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.954+0000 7f28527fc700 1 -- 172.21.15.47:0/2343457538 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f285801a9c0 con 0x7f285c1070a0 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.955+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 msgr2=0x7f285c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.955+0000 7f2863314700 1 --2- 172.21.15.47:0/2343457538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1099a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f28580117b0 tx=0x7f2858011b70 comp rx=0 tx=0).stop 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.955+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 shutdown_connections 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.955+0000 7f2863314700 1 --2- 172.21.15.47:0/2343457538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f285c1070a0 0x7f285c1099a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.955+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 >> 172.21.15.47:0/2343457538 conn(0x7f285c1029c0 msgr2=0x7f285c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.955+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 shutdown_connections 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:53.955+0000 7f2863314700 1 -- 172.21.15.47:0/2343457538 wait complete. 2024-05-11T12:12:54.647 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (1/15)... 2024-05-11T12:12:55.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:55.453+0000 7f9b16cfe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T12:12:55.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:55.630+0000 7f9b16cfe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T12:12:56.107 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:55.808+0000 7f9b16cfe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T12:12:56.107 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:56.107+0000 7f9b16cfe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T12:12:56.416 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:56.204+0000 7f9b16cfe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T12:12:56.416 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:56.416+0000 7f9b16cfe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T12:12:56.764 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:56.618+0000 7f9b16cfe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:12:57.497 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:12:57 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2718254388' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T12:12:57.497 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:57.265+0000 7f9b16cfe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T12:12:57.774 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:57.499+0000 7f9b16cfe200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T12:12:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:12:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:12:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "9b1a9a48-0f8f-11ef-bc98-c7b262605968", 2024-05-11T12:12:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T12:12:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T12:12:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T12:12:57.811 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T12:12:57.812 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T12:12:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T12:12:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T12:12:57.813 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T12:12:57.821 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T12:12:57.822 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:57.823 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T12:12:45.658187+0000", 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.099+0000 7fa87ab06700 1 Processor -- start 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.099+0000 7fa87ab06700 1 -- start start 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.099+0000 7fa87ab06700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87410bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.099+0000 7fa87ab06700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa87407ce40 con 0x7fa874109830 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8788a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87410bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8788a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87410bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49630/0 (socket says 172.21.15.47:49630) 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8788a2700 1 -- 172.21.15.47:0/3881177708 learned_addr learned my addr 172.21.15.47:0/3881177708 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:57.824 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8788a2700 1 -- 172.21.15.47:0/3881177708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87410c160 con 0x7fa874109830 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8788a2700 1 --2- 172.21.15.47:0/3881177708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87410bc20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8680098a0 tx=0x7fa868009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d9af8a95c2f260 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8737fe700 1 -- 172.21.15.47:0/3881177708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa868004160 con 0x7fa874109830 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8737fe700 1 -- 172.21.15.47:0/3881177708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa868004300 con 0x7fa874109830 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.100+0000 7fa8737fe700 1 -- 172.21.15.47:0/3881177708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa868011af0 con 0x7fa874109830 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.101+0000 7fa87ab06700 1 -- 172.21.15.47:0/3881177708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 msgr2=0x7fa87410bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.101+0000 7fa87ab06700 1 --2- 172.21.15.47:0/3881177708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87410bc20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8680098a0 tx=0x7fa868009bb0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.101+0000 7fa87ab06700 1 -- 172.21.15.47:0/3881177708 shutdown_connections 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.101+0000 7fa87ab06700 1 --2- 172.21.15.47:0/3881177708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87410bc20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.101+0000 7fa87ab06700 1 -- 172.21.15.47:0/3881177708 >> 172.21.15.47:0/3881177708 conn(0x7fa874103240 msgr2=0x7fa8741056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.101+0000 7fa87ab06700 1 -- 172.21.15.47:0/3881177708 shutdown_connections 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.101+0000 7fa87ab06700 1 -- 172.21.15.47:0/3881177708 wait complete. 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.102+0000 7fa87ab06700 1 Processor -- start 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.102+0000 7fa87ab06700 1 -- start start 2024-05-11T12:12:57.825 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.102+0000 7fa87ab06700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.102+0000 7fa87ab06700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa87410d130 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.102+0000 7fa8788a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.102+0000 7fa8788a2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49642/0 (socket says 172.21.15.47:49642) 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.102+0000 7fa8788a2700 1 -- 172.21.15.47:0/2718254388 learned_addr learned my addr 172.21.15.47:0/2718254388 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa8788a2700 1 -- 172.21.15.47:0/2718254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87419d2a0 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa8788a2700 1 --2- 172.21.15.47:0/2718254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87419cd60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa868011ac0 tx=0x7fa868011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa871ffb700 1 -- 172.21.15.47:0/2718254388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa868003970 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa871ffb700 1 -- 172.21.15.47:0/2718254388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa868003b10 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87419d530 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa871ffb700 1 -- 172.21.15.47:0/2718254388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa868024610 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741a0220 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa871ffb700 1 -- 172.21.15.47:0/2718254388 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa868024c20 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.103+0000 7fa871ffb700 1 -- 172.21.15.47:0/2718254388 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa868019700 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.104+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa87419da10 con 0x7fa874109830 2024-05-11T12:12:57.826 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.106+0000 7fa871ffb700 1 -- 172.21.15.47:0/2718254388 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa868043ec0 con 0x7fa874109830 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.183+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa8741a03f0 con 0x7fa874109830 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.183+0000 7fa871ffb700 1 -- 172.21.15.47:0/2718254388 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa868032030 con 0x7fa874109830 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.184+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 msgr2=0x7fa87419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.184+0000 7fa87ab06700 1 --2- 172.21.15.47:0/2718254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87419cd60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa868011ac0 tx=0x7fa868011ba0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.184+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 shutdown_connections 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.184+0000 7fa87ab06700 1 --2- 172.21.15.47:0/2718254388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa874109830 0x7fa87419cd60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.184+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 >> 172.21.15.47:0/2718254388 conn(0x7fa874103240 msgr2=0x7fa874103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.184+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 shutdown_connections 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:12:57.184+0000 7fa87ab06700 1 -- 172.21.15.47:0/2718254388 wait complete. 2024-05-11T12:12:57.827 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (2/15)... 2024-05-11T12:12:58.274 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:57.942+0000 7f9b16cfe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T12:12:58.275 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:58.042+0000 7f9b16cfe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T12:12:58.774 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:58.427+0000 7f9b16cfe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T12:12:59.151 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:58.800+0000 7f9b16cfe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T12:12:59.151 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:58.897+0000 7f9b16cfe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T12:12:59.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:59.151+0000 7f9b16cfe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T12:12:59.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:59.291+0000 7f9b16cfe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T12:12:59.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:59.390+0000 7f9b16cfe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T12:13:00.025 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:59.731+0000 7f9b16cfe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T12:13:00.026 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:12:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:12:59.828+0000 7f9b16cfe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T12:13:00.620 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:00 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3565527270' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T12:13:00.620 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:00.502+0000 7f9b16cfe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T12:13:00.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:13:00.866 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "9b1a9a48-0f8f-11ef-bc98-c7b262605968", 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-11T12:13:00.867 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T12:13:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T12:13:00.876 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T12:13:00.877 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T12:12:45.658187+0000", 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.245+0000 7fd89e1ee700 1 Processor -- start 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.245+0000 7fd89e1ee700 1 -- start start 2024-05-11T12:13:00.878 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.245+0000 7fd89e1ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd8981080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.245+0000 7fd89e1ee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd8981086b0 con 0x7fd898107cc0 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.245+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd8981080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.245+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd8981080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49646/0 (socket says 172.21.15.47:49646) 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.245+0000 7fd8977fe700 1 -- 172.21.15.47:0/3710847702 learned_addr learned my addr 172.21.15.47:0/3710847702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.246+0000 7fd8977fe700 1 -- 172.21.15.47:0/3710847702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd898108ec0 con 0x7fd898107cc0 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.246+0000 7fd8977fe700 1 --2- 172.21.15.47:0/3710847702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd8981080e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd88800bb40 tx=0x7fd88800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99bfd2c99a853a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.246+0000 7fd8967fc700 1 -- 172.21.15.47:0/3710847702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd88800dbe0 con 0x7fd898107cc0 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.246+0000 7fd8967fc700 1 -- 172.21.15.47:0/3710847702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd8880044c0 con 0x7fd898107cc0 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.246+0000 7fd8967fc700 1 -- 172.21.15.47:0/3710847702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd888010470 con 0x7fd898107cc0 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3710847702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 msgr2=0x7fd8981080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 --2- 172.21.15.47:0/3710847702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd8981080e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd88800bb40 tx=0x7fd88800be50 comp rx=0 tx=0).stop 2024-05-11T12:13:00.879 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3710847702 shutdown_connections 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 --2- 172.21.15.47:0/3710847702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd8981080e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3710847702 >> 172.21.15.47:0/3710847702 conn(0x7fd898103240 msgr2=0x7fd8981056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3710847702 shutdown_connections 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3710847702 wait complete. 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.247+0000 7fd89e1ee700 1 Processor -- start 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd89e1ee700 1 -- start start 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd89e1ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd89819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd89e1ee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd89807bf60 con 0x7fd898107cc0 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd89819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd89819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49654/0 (socket says 172.21.15.47:49654) 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd8977fe700 1 -- 172.21.15.47:0/3565527270 learned_addr learned my addr 172.21.15.47:0/3565527270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd8977fe700 1 -- 172.21.15.47:0/3565527270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89819b060 con 0x7fd898107cc0 2024-05-11T12:13:00.880 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.248+0000 7fd8977fe700 1 --2- 172.21.15.47:0/3565527270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd89819ab20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd88800b340 tx=0x7fd888003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.249+0000 7fd894ff9700 1 -- 172.21.15.47:0/3565527270 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd888027070 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.249+0000 7fd894ff9700 1 -- 172.21.15.47:0/3565527270 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd888010bb0 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.249+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89819b2f0 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.249+0000 7fd894ff9700 1 -- 172.21.15.47:0/3565527270 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd888022ba0 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.249+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89819dfe0 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.249+0000 7fd894ff9700 1 -- 172.21.15.47:0/3565527270 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd88802a070 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.249+0000 7fd894ff9700 1 -- 172.21.15.47:0/3565527270 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd88801f070 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.250+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd898064bb0 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.252+0000 7fd894ff9700 1 -- 172.21.15.47:0/3565527270 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd888031030 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.328+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd898194080 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.328+0000 7fd894ff9700 1 -- 172.21.15.47:0/3565527270 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd888015070 con 0x7fd898107cc0 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.329+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 msgr2=0x7fd89819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.329+0000 7fd89e1ee700 1 --2- 172.21.15.47:0/3565527270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd89819ab20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd88800b340 tx=0x7fd888003dc0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.881 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.329+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 shutdown_connections 2024-05-11T12:13:00.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.329+0000 7fd89e1ee700 1 --2- 172.21.15.47:0/3565527270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd898107cc0 0x7fd89819ab20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:00.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.329+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 >> 172.21.15.47:0/3565527270 conn(0x7fd898103240 msgr2=0x7fd898191140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:00.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.330+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 shutdown_connections 2024-05-11T12:13:00.882 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:00.330+0000 7fd89e1ee700 1 -- 172.21.15.47:0/3565527270 wait complete. 2024-05-11T12:13:00.882 INFO:teuthology.orchestra.run.smithi047.stdout:mgr not available, waiting (3/15)... 2024-05-11T12:13:01.024 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:00.730+0000 7f9b16cfe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T12:13:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: Activating manager daemon x 2024-05-11T12:13:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: mgrmap e2: x(active, starting, since 0.00331879s) 2024-05-11T12:13:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T12:13:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T12:13:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T12:13:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:13:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T12:13:01.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: Manager daemon x is now available 2024-05-11T12:13:01.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T12:13:01.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T12:13:01.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' 2024-05-11T12:13:01.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' 2024-05-11T12:13:01.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:01 smithi047 ceph-mon[30375]: from='mgr.14100 172.21.15.47:0/1850127823' entity='mgr.x' 2024-05-11T12:13:02.996 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:02 smithi047 ceph-mon[30375]: mgrmap e3: x(active, since 1.00725s) 2024-05-11T12:13:03.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:03 smithi047 ceph-mon[30375]: mgrmap e4: x(active, since 2s) 2024-05-11T12:13:03.999 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:03 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1967118279' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsid": "9b1a9a48-0f8f-11ef-bc98-c7b262605968", 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "health": { 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:04.225 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 0 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "a" 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:04.226 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-11T12:13:04.227 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-11T12:13:04.236 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "restful" 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ], 2024-05-11T12:13:04.237 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "modified": "2024-05-11T12:12:45.658187+0000", 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout }, 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.296+0000 7f5af8c73700 1 Processor -- start 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.296+0000 7f5af8c73700 1 -- start start 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.296+0000 7f5af8c73700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af4109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.296+0000 7f5af8c73700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5af4109790 con 0x7f5af4108e30 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af4109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.238 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af4109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49736/0 (socket says 172.21.15.47:49736) 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af259c700 1 -- 172.21.15.47:0/2707023236 learned_addr learned my addr 172.21.15.47:0/2707023236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af259c700 1 -- 172.21.15.47:0/2707023236 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af4109f90 con 0x7f5af4108e30 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af259c700 1 --2- 172.21.15.47:0/2707023236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af4109250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098a0 tx=0x7f5ae4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45d9734a77027b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af159a700 1 -- 172.21.15.47:0/2707023236 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae4004160 con 0x7f5af4108e30 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af159a700 1 -- 172.21.15.47:0/2707023236 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5ae4004300 con 0x7f5af4108e30 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.297+0000 7f5af159a700 1 -- 172.21.15.47:0/2707023236 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae4011af0 con 0x7f5af4108e30 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.298+0000 7f5af8c73700 1 -- 172.21.15.47:0/2707023236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 msgr2=0x7f5af4109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.298+0000 7f5af8c73700 1 --2- 172.21.15.47:0/2707023236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af4109250 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098a0 tx=0x7f5ae4009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.298+0000 7f5af8c73700 1 -- 172.21.15.47:0/2707023236 shutdown_connections 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.298+0000 7f5af8c73700 1 --2- 172.21.15.47:0/2707023236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af4109250 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.298+0000 7f5af8c73700 1 -- 172.21.15.47:0/2707023236 >> 172.21.15.47:0/2707023236 conn(0x7f5af41043f0 msgr2=0x7f5af4106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.298+0000 7f5af8c73700 1 -- 172.21.15.47:0/2707023236 shutdown_connections 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.298+0000 7f5af8c73700 1 -- 172.21.15.47:0/2707023236 wait complete. 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.299+0000 7f5af8c73700 1 Processor -- start 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.299+0000 7f5af8c73700 1 -- start start 2024-05-11T12:13:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.299+0000 7f5af8c73700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.299+0000 7f5af8c73700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5af410d3d0 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.299+0000 7f5af259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.299+0000 7f5af259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49744/0 (socket says 172.21.15.47:49744) 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.299+0000 7f5af259c700 1 -- 172.21.15.47:0/1967118279 learned_addr learned my addr 172.21.15.47:0/1967118279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.300+0000 7f5af259c700 1 -- 172.21.15.47:0/1967118279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af419d260 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.300+0000 7f5af259c700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af419cd20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4003f40 tx=0x7f5ae4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.300+0000 7f5ae37fe700 1 -- 172.21.15.47:0/1967118279 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae40038b0 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.300+0000 7f5ae37fe700 1 -- 172.21.15.47:0/1967118279 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5ae4003a50 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.300+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af419d4f0 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.300+0000 7f5ae37fe700 1 -- 172.21.15.47:0/1967118279 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ae401a5c0 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.300+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af41a01d0 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.301+0000 7f5ae37fe700 1 -- 172.21.15.47:0/1967118279 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f5ae4024430 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.301+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af4064bb0 con 0x7f5af4108e30 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.302+0000 7f5ae37fe700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f5adc03e3d0 0x7f5adc040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:04.240 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.302+0000 7f5ae37fe700 1 -- 172.21.15.47:0/1967118279 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ae4053a70 con 0x7f5af4108e30 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.302+0000 7f5af1d9b700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f5adc03e3d0 0x7f5adc040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.302+0000 7f5af1d9b700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f5adc03e3d0 0x7f5adc040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006fd0 tx=0x7f5ae8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.307+0000 7f5ae37fe700 1 -- 172.21.15.47:0/1967118279 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ae400b3e0 con 0x7f5af4108e30 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.616+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5af41a03a0 con 0x7f5af4108e30 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.617+0000 7f5ae37fe700 1 -- 172.21.15.47:0/1967118279 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5af41a03a0 con 0x7f5af4108e30 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f5adc03e3d0 msgr2=0x7f5adc040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f5adc03e3d0 0x7f5adc040890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8006fd0 tx=0x7f5ae8006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 msgr2=0x7f5af419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af419cd20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4003f40 tx=0x7f5ae4011890 comp rx=0 tx=0).stop 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 shutdown_connections 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f5adc03e3d0 0x7f5adc040890 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 --2- 172.21.15.47:0/1967118279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af4108e30 0x7f5af419cd20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 >> 172.21.15.47:0/1967118279 conn(0x7f5af41043f0 msgr2=0x7f5af4104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 shutdown_connections 2024-05-11T12:13:04.241 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:03.619+0000 7f5af8c73700 1 -- 172.21.15.47:0/1967118279 wait complete. 2024-05-11T12:13:04.242 INFO:teuthology.orchestra.run.smithi047.stdout:mgr is available 2024-05-11T12:13:05.224 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:04 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/545185371' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-11T12:13:05.509 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [global] 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout fsid = 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 2024-05-11T12:13:05.510 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout [osd] 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.695+0000 7f9cadcef700 1 Processor -- start 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.695+0000 7f9cadcef700 1 -- start start 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.695+0000 7f9cadcef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca8109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.695+0000 7f9cadcef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca8109790 con 0x7f9ca8108e30 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.695+0000 7f9ca77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca8109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.695+0000 7f9ca77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca8109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49748/0 (socket says 172.21.15.47:49748) 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.695+0000 7f9ca77fe700 1 -- 172.21.15.47:0/1890792924 learned_addr learned my addr 172.21.15.47:0/1890792924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.696+0000 7f9ca77fe700 1 -- 172.21.15.47:0/1890792924 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca8109f90 con 0x7f9ca8108e30 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.696+0000 7f9ca77fe700 1 --2- 172.21.15.47:0/1890792924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca8109250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9c980098a0 tx=0x7f9c98009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f2dcaf892451ab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.696+0000 7f9ca6ffd700 1 -- 172.21.15.47:0/1890792924 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c98004160 con 0x7f9ca8108e30 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.696+0000 7f9ca6ffd700 1 -- 172.21.15.47:0/1890792924 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c98004300 con 0x7f9ca8108e30 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.696+0000 7f9ca6ffd700 1 -- 172.21.15.47:0/1890792924 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c98011af0 con 0x7f9ca8108e30 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 -- 172.21.15.47:0/1890792924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 msgr2=0x7f9ca8109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 --2- 172.21.15.47:0/1890792924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca8109250 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9c980098a0 tx=0x7f9c98009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 -- 172.21.15.47:0/1890792924 shutdown_connections 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 --2- 172.21.15.47:0/1890792924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca8109250 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 -- 172.21.15.47:0/1890792924 >> 172.21.15.47:0/1890792924 conn(0x7f9ca81043f0 msgr2=0x7f9ca8106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 -- 172.21.15.47:0/1890792924 shutdown_connections 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 -- 172.21.15.47:0/1890792924 wait complete. 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.697+0000 7f9cadcef700 1 Processor -- start 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9cadcef700 1 -- start start 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9cadcef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.512 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9cadcef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca810d3d0 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9ca77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9ca77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca819cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49752/0 (socket says 172.21.15.47:49752) 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9ca77fe700 1 -- 172.21.15.47:0/545185371 learned_addr learned my addr 172.21.15.47:0/545185371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9ca77fe700 1 -- 172.21.15.47:0/545185371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca819d260 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.698+0000 7f9ca77fe700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca819cd20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9c98003f40 tx=0x7f9c98011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.699+0000 7f9ca57fa700 1 -- 172.21.15.47:0/545185371 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c980038b0 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.699+0000 7f9ca57fa700 1 -- 172.21.15.47:0/545185371 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c98003a50 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.699+0000 7f9ca57fa700 1 -- 172.21.15.47:0/545185371 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c9801a5c0 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.699+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca819d4f0 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.699+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca81a01d0 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.700+0000 7f9ca57fa700 1 -- 172.21.15.47:0/545185371 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f9c98024430 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.700+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ca8064bb0 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.700+0000 7f9ca57fa700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f9c9403e3d0 0x7f9c94040890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.700+0000 7f9ca57fa700 1 -- 172.21.15.47:0/545185371 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c98053a70 con 0x7f9ca8108e30 2024-05-11T12:13:05.513 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.700+0000 7f9c9effd700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f9c9403e3d0 0x7f9c94040890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.701+0000 7f9c9effd700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f9c9403e3d0 0x7f9c94040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.705+0000 7f9ca57fa700 1 -- 172.21.15.47:0/545185371 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9c9800b3e0 con 0x7f9ca8108e30 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.921+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f9ca81a03a0 con 0x7f9ca8108e30 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.922+0000 7f9ca57fa700 1 -- 172.21.15.47:0/545185371 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f9ca81a03a0 con 0x7f9ca8108e30 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f9c9403e3d0 msgr2=0x7f9c94040890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f9c9403e3d0 0x7f9c94040890 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 msgr2=0x7f9ca819cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca819cd20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9c98003f40 tx=0x7f9c98011890 comp rx=0 tx=0).stop 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 shutdown_connections 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f9c9403e3d0 0x7f9c94040890 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 --2- 172.21.15.47:0/545185371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ca8108e30 0x7f9ca819cd20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 >> 172.21.15.47:0/545185371 conn(0x7f9ca81043f0 msgr2=0x7f9ca8104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:05.514 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 shutdown_connections 2024-05-11T12:13:05.515 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:04.925+0000 7f9cadcef700 1 -- 172.21.15.47:0/545185371 wait complete. 2024-05-11T12:13:05.515 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling cephadm module... 2024-05-11T12:13:06.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:06 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3756667694' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-11T12:13:06.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: ignoring --setuser ceph since I am not root 2024-05-11T12:13:06.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: ignoring --setgroup ceph since I am not root 2024-05-11T12:13:06.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:06.479+0000 7f5b5edb3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T12:13:07.024 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:06.671+0000 7f5b5edb3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:13:07.025 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:06.782+0000 7f5b5edb3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T12:13:07.098 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7f25f700 1 Processor -- start 2024-05-11T12:13:07.098 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7f25f700 1 -- start start 2024-05-11T12:13:07.098 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7f25f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b78109230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.098 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7f25f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78109770 con 0x7f4b78108e10 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7cffb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b78109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7cffb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b78109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49756/0 (socket says 172.21.15.47:49756) 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7cffb700 1 -- 172.21.15.47:0/3017518384 learned_addr learned my addr 172.21.15.47:0/3017518384 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7cffb700 1 -- 172.21.15.47:0/3017518384 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b78109f70 con 0x7f4b78108e10 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.961+0000 7f4b7cffb700 1 --2- 172.21.15.47:0/3017518384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b78109230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4b680098a0 tx=0x7f4b68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17ae3a9b0095554 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b777fe700 1 -- 172.21.15.47:0/3017518384 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68004160 con 0x7f4b78108e10 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b777fe700 1 -- 172.21.15.47:0/3017518384 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b68004300 con 0x7f4b78108e10 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b777fe700 1 -- 172.21.15.47:0/3017518384 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68011af0 con 0x7f4b78108e10 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3017518384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 msgr2=0x7f4b78109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b7f25f700 1 --2- 172.21.15.47:0/3017518384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b78109230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4b680098a0 tx=0x7f4b68009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3017518384 shutdown_connections 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b7f25f700 1 --2- 172.21.15.47:0/3017518384 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b78109230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.962+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3017518384 >> 172.21.15.47:0/3017518384 conn(0x7f4b781043f0 msgr2=0x7f4b78106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:07.099 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3017518384 shutdown_connections 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3017518384 wait complete. 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7f25f700 1 Processor -- start 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7f25f700 1 -- start start 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7f25f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b7819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7f25f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7810d3b0 con 0x7f4b78108e10 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7cffb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b7819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.963+0000 7f4b7cffb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b7819cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49760/0 (socket says 172.21.15.47:49760) 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b7cffb700 1 -- 172.21.15.47:0/3756667694 learned_addr learned my addr 172.21.15.47:0/3756667694 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b7cffb700 1 -- 172.21.15.47:0/3756667694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7819d330 con 0x7f4b78108e10 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b7cffb700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b7819cdf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b68011ac0 tx=0x7f4b68011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68003970 con 0x7f4b78108e10 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b68003b10 con 0x7f4b78108e10 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7819d5c0 con 0x7f4b78108e10 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b68024610 con 0x7f4b78108e10 2024-05-11T12:13:07.100 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.964+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b781a02b0 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.965+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b78064bb0 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.965+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f4b68024840 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.966+0000 7f4b75ffb700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f4b6003e470 0x7f4b60040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.966+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b68053950 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.966+0000 7f4b77fff700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f4b6003e470 0x7f4b60040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.966+0000 7f4b77fff700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f4b6003e470 0x7f4b60040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c006fd0 tx=0x7f4b6c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:05.971+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b6800a0d0 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.245+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f4b781a0480 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.296+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f4b781a0480 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.298+0000 7f4b75ffb700 1 -- 172.21.15.47:0/3756667694 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f4b68019cd0 con 0x7f4b78108e10 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f4b6003e470 msgr2=0x7f4b60040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f4b6003e470 0x7f4b60040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c006fd0 tx=0x7f4b6c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 msgr2=0x7f4b7819cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:07.101 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b7819cdf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4b68011ac0 tx=0x7f4b68011ba0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.102 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 shutdown_connections 2024-05-11T12:13:07.102 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f4b6003e470 0x7f4b60040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.102 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 --2- 172.21.15.47:0/3756667694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b78108e10 0x7f4b7819cdf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:07.102 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.299+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 >> 172.21.15.47:0/3756667694 conn(0x7f4b781043f0 msgr2=0x7f4b78104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:07.102 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.300+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 shutdown_connections 2024-05-11T12:13:07.102 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:06.300+0000 7f4b7f25f700 1 -- 172.21.15.47:0/3756667694 wait complete. 2024-05-11T12:13:07.409 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:07 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3756667694' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-11T12:13:07.410 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:07 smithi047 ceph-mon[30375]: mgrmap e5: x(active, since 5s) 2024-05-11T12:13:07.410 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:07.106+0000 7f5b5edb3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T12:13:07.410 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:07.203+0000 7f5b5edb3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T12:13:08.571 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:08 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3147527508' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-11T12:13:08.571 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:08 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:08.462+0000 7f5b5edb3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.513+0000 7f478a072700 1 Processor -- start 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.513+0000 7f478a072700 1 -- start start 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f478a072700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478410bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f478a072700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f478407cd80 con 0x7f47841098f0 2024-05-11T12:13:08.580 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f47837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478410bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f47837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478410bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34122/0 (socket says 172.21.15.47:34122) 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f47837fe700 1 -- 172.21.15.47:0/892258571 learned_addr learned my addr 172.21.15.47:0/892258571 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f47837fe700 1 -- 172.21.15.47:0/892258571 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478410c220 con 0x7f47841098f0 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f47837fe700 1 --2- 172.21.15.47:0/892258571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478410bce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47740098a0 tx=0x7f4774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d84fef356e90f4a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.514+0000 7f47827fc700 1 -- 172.21.15.47:0/892258571 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4774004160 con 0x7f47841098f0 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f47827fc700 1 -- 172.21.15.47:0/892258571 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4774004300 con 0x7f47841098f0 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f47827fc700 1 -- 172.21.15.47:0/892258571 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4774011af0 con 0x7f47841098f0 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f478a072700 1 -- 172.21.15.47:0/892258571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 msgr2=0x7f478410bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f478a072700 1 --2- 172.21.15.47:0/892258571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478410bce0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f47740098a0 tx=0x7f4774009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f478a072700 1 -- 172.21.15.47:0/892258571 shutdown_connections 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f478a072700 1 --2- 172.21.15.47:0/892258571 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478410bce0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f478a072700 1 -- 172.21.15.47:0/892258571 >> 172.21.15.47:0/892258571 conn(0x7f4784103240 msgr2=0x7f47841056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f478a072700 1 -- 172.21.15.47:0/892258571 shutdown_connections 2024-05-11T12:13:08.581 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.515+0000 7f478a072700 1 -- 172.21.15.47:0/892258571 wait complete. 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.516+0000 7f478a072700 1 Processor -- start 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.516+0000 7f478a072700 1 -- start start 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.516+0000 7f478a072700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.516+0000 7f478a072700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f478410d130 con 0x7f47841098f0 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.516+0000 7f47837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.516+0000 7f47837fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34124/0 (socket says 172.21.15.47:34124) 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.516+0000 7f47837fe700 1 -- 172.21.15.47:0/3147527508 learned_addr learned my addr 172.21.15.47:0/3147527508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.517+0000 7f47837fe700 1 -- 172.21.15.47:0/3147527508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478419d2a0 con 0x7f47841098f0 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.517+0000 7f47837fe700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478419cd60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4774011ac0 tx=0x7f4774011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.517+0000 7f4780ff9700 1 -- 172.21.15.47:0/3147527508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4774003970 con 0x7f47841098f0 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.517+0000 7f4780ff9700 1 -- 172.21.15.47:0/3147527508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4774003b10 con 0x7f47841098f0 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.517+0000 7f4780ff9700 1 -- 172.21.15.47:0/3147527508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4774024610 con 0x7f47841098f0 2024-05-11T12:13:08.582 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.517+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478419d530 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.517+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47841a0220 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.518+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4784064bb0 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.518+0000 7f4780ff9700 1 -- 172.21.15.47:0/3147527508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f4774024840 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.519+0000 7f4780ff9700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 0x7f4770040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.519+0000 7f4780ff9700 1 -- 172.21.15.47:0/3147527508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f47740539a0 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.519+0000 7f4782ffd700 1 -- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 msgr2=0x7f4770040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2872683180 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.519+0000 7f4782ffd700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 0x7f4770040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.524+0000 7f4780ff9700 1 -- 172.21.15.47:0/3147527508 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f477400a0d0 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.720+0000 7f4782ffd700 1 -- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 msgr2=0x7f4770040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2872683180 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.720+0000 7f4782ffd700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 0x7f4770040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.845+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f47841a03f0 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.846+0000 7f4780ff9700 1 -- 172.21.15.47:0/3147527508 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f47841a03f0 con 0x7f47841098f0 2024-05-11T12:13:08.583 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 msgr2=0x7f4770040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 0x7f4770040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 msgr2=0x7f478419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478419cd60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4774011ac0 tx=0x7f4774011ba0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 shutdown_connections 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f477003e4c0 0x7f4770040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 --2- 172.21.15.47:0/3147527508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47841098f0 0x7f478419cd60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 >> 172.21.15.47:0/3147527508 conn(0x7f4784103240 msgr2=0x7f4784103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 shutdown_connections 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:07.848+0000 7f478a072700 1 -- 172.21.15.47:0/3147527508 wait complete. 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-05-11T12:13:08.584 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 5... 2024-05-11T12:13:08.824 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:08 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:08.644+0000 7f5b5edb3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T12:13:08.825 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:08 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:08.825+0000 7f5b5edb3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T12:13:09.438 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:09 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:09.131+0000 7f5b5edb3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T12:13:09.438 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:09 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:09.227+0000 7f5b5edb3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T12:13:09.438 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:09 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:09.438+0000 7f5b5edb3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T12:13:09.774 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:09 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:09.639+0000 7f5b5edb3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:13:10.524 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:10 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:10.268+0000 7f5b5edb3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T12:13:10.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:10 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:10.499+0000 7f5b5edb3200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T12:13:11.275 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:10 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:10.940+0000 7f5b5edb3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T12:13:11.275 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:11 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:11.040+0000 7f5b5edb3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T12:13:11.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:11 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:11.424+0000 7f5b5edb3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T12:13:12.148 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:11 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:11.797+0000 7f5b5edb3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T12:13:12.148 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:11 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:11.894+0000 7f5b5edb3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T12:13:12.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:12 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:12.148+0000 7f5b5edb3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T12:13:12.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:12 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:12.288+0000 7f5b5edb3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T12:13:12.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:12 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:12.391+0000 7f5b5edb3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T12:13:12.837 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:12 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:12.738+0000 7f5b5edb3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T12:13:13.275 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:12 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:12.837+0000 7f5b5edb3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T12:13:13.755 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:13 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:13.475+0000 7f5b5edb3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T12:13:13.755 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:13 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:13.702+0000 7f5b5edb3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: Active manager daemon x restarted 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: Activating manager daemon x 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: osdmap e2: 0 total, 0 up, 0 in 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: mgrmap e6: x(active, starting, since 0.00332221s) 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T12:13:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T12:13:14.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: Manager daemon x is now available 2024-05-11T12:13:14.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:14.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:14.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:14.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:14.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:13 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T12:13:15.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:14 smithi047 ceph-mon[30375]: Found migration_current of "None". Setting to last migration. 2024-05-11T12:13:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:14 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T12:13:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:14 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:14 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:14 smithi047 ceph-mon[30375]: mgrmap e7: x(active, since 1.00562s) 2024-05-11T12:13:15.330 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:13:15.330 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.970+0000 7f7521f04700 1 Processor -- start 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.970+0000 7f7521f04700 1 -- start start 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.970+0000 7f7521f04700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.970+0000 7f7521f04700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1097e0 con 0x7f751c108df0 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.970+0000 7f751b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.970+0000 7f751b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34138/0 (socket says 172.21.15.47:34138) 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.970+0000 7f751b7fe700 1 -- 172.21.15.47:0/3418723522 learned_addr learned my addr 172.21.15.47:0/3418723522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.971+0000 7f751b7fe700 1 -- 172.21.15.47:0/3418723522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c109ff0 con 0x7f751c108df0 2024-05-11T12:13:15.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.971+0000 7f751b7fe700 1 --2- 172.21.15.47:0/3418723522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c109210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f750c0098a0 tx=0x7f750c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa1a489fbece20c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.971+0000 7f751a7fc700 1 -- 172.21.15.47:0/3418723522 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f750c004160 con 0x7f751c108df0 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.971+0000 7f751a7fc700 1 -- 172.21.15.47:0/3418723522 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f750c004300 con 0x7f751c108df0 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.971+0000 7f751a7fc700 1 -- 172.21.15.47:0/3418723522 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f750c011af0 con 0x7f751c108df0 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 -- 172.21.15.47:0/3418723522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 msgr2=0x7f751c109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 --2- 172.21.15.47:0/3418723522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c109210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f750c0098a0 tx=0x7f750c009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 -- 172.21.15.47:0/3418723522 shutdown_connections 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 --2- 172.21.15.47:0/3418723522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c109210 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 -- 172.21.15.47:0/3418723522 >> 172.21.15.47:0/3418723522 conn(0x7f751c104390 msgr2=0x7f751c1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 -- 172.21.15.47:0/3418723522 shutdown_connections 2024-05-11T12:13:15.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 -- 172.21.15.47:0/3418723522 wait complete. 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.972+0000 7f7521f04700 1 Processor -- start 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f7521f04700 1 -- start start 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f7521f04700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f7521f04700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f751c10d3a0 con 0x7f751c108df0 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f751b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f751b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c19ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34150/0 (socket says 172.21.15.47:34150) 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f751b7fe700 1 -- 172.21.15.47:0/5302006 learned_addr learned my addr 172.21.15.47:0/5302006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f751b7fe700 1 -- 172.21.15.47:0/5302006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c19f4b0 con 0x7f751c108df0 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f751b7fe700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c19ef70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f750c000c00 tx=0x7f750c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.973+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f750c003b20 con 0x7f751c108df0 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.974+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f750c01a8c0 con 0x7f751c108df0 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.974+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c19f740 con 0x7f751c108df0 2024-05-11T12:13:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.974+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f750c024940 con 0x7f751c108df0 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.974+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c1a2430 con 0x7f751c108df0 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.975+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f750c022020 con 0x7f751c108df0 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.975+0000 7f7518ff9700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.975+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 --> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7504041090 con 0x7f750403e4c0 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.975+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f750c053530 con 0x7f751c108df0 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.975+0000 7f751affd700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 msgr2=0x7f7504040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2872683180 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:08.975+0000 7f751affd700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:09.176+0000 7f751affd700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 msgr2=0x7f7504040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2872683180 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:09.176+0000 7f751affd700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:09.577+0000 7f751affd700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 msgr2=0x7f7504040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2872683180 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:09.577+0000 7f751affd700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:10.378+0000 7f751affd700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 msgr2=0x7f7504040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2872683180 2024-05-11T12:13:15.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:10.378+0000 7f751affd700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:11.979+0000 7f751affd700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 msgr2=0x7f7504040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/2872683180 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:11.979+0000 7f751affd700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:13.705+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f750c01aae0 con 0x7f751c108df0 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:13.705+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 msgr2=0x7f7504040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:13.705+0000 7f7518ff9700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.707+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f750c055040 con 0x7f751c108df0 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.707+0000 7f7518ff9700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7504041db0 0x7f75040441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.707+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f750c025e70 con 0x7f7504041db0 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.707+0000 7f751affd700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7504041db0 0x7f75040441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.708+0000 7f751affd700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7504041db0 0x7f75040441a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7510003a10 tx=0x7f75100092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.708+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f750c025e70 con 0x7f7504041db0 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.717+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f751c06a640 con 0x7f7504041db0 2024-05-11T12:13:15.335 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.717+0000 7f7518ff9700 1 -- 172.21.15.47:0/5302006 <== mgr.14120 v2:172.21.15.47:6800/3174346748 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f751c06a640 con 0x7f7504041db0 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.717+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7504041db0 msgr2=0x7f75040441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.717+0000 7f7521f04700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7504041db0 0x7f75040441a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7510003a10 tx=0x7f75100092b0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.717+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 msgr2=0x7f751c19ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.717+0000 7f7521f04700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c19ef70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f750c000c00 tx=0x7f750c011b70 comp rx=0 tx=0).stop 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.718+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 shutdown_connections 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.718+0000 7f7521f04700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7504041db0 0x7f75040441a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.718+0000 7f7521f04700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f751c108df0 0x7f751c19ef70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.718+0000 7f7521f04700 1 --2- 172.21.15.47:0/5302006 >> [v2:172.21.15.47:6800/2872683180,v1:172.21.15.47:6801/2872683180] conn(0x7f750403e4c0 0x7f7504040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.718+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 >> 172.21.15.47:0/5302006 conn(0x7f751c104390 msgr2=0x7f751c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.718+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 shutdown_connections 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:14.718+0000 7f7521f04700 1 -- 172.21.15.47:0/5302006 wait complete. 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 5 is available 2024-05-11T12:13:15.336 INFO:teuthology.orchestra.run.smithi047.stdout:Setting orchestrator backend to cephadm... 2024-05-11T12:13:15.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:15 smithi047 ceph-mon[30375]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-11T12:13:15.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:15 smithi047 ceph-mon[30375]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-11T12:13:15.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:15 smithi047 ceph-mon[30375]: [11/May/2024:12:13:14] ENGINE Bus STARTING 2024-05-11T12:13:15.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:15 smithi047 ceph-mon[30375]: [11/May/2024:12:13:14] ENGINE Serving on http://172.21.15.47:8765 2024-05-11T12:13:15.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:15 smithi047 ceph-mon[30375]: [11/May/2024:12:13:15] ENGINE Serving on https://172.21.15.47:7150 2024-05-11T12:13:15.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:15 smithi047 ceph-mon[30375]: [11/May/2024:12:13:15] ENGINE Bus STARTED 2024-05-11T12:13:15.985 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:15 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.745+0000 7f1753ab7700 1 Processor -- start 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.745+0000 7f1753ab7700 1 -- start start 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.745+0000 7f1753ab7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c10bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.745+0000 7f1753ab7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f174c07d670 con 0x7f174c109900 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.745+0000 7f1751853700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c10bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.746+0000 7f1751853700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c10bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34224/0 (socket says 172.21.15.47:34224) 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.746+0000 7f1751853700 1 -- 172.21.15.47:0/961090664 learned_addr learned my addr 172.21.15.47:0/961090664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.746+0000 7f1751853700 1 -- 172.21.15.47:0/961090664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c10c230 con 0x7f174c109900 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.746+0000 7f1751853700 1 --2- 172.21.15.47:0/961090664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c10bcf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17480098a0 tx=0x7f1748009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400d33ef45659862 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.746+0000 7f1750851700 1 -- 172.21.15.47:0/961090664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1748004160 con 0x7f174c109900 2024-05-11T12:13:16.537 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.746+0000 7f1750851700 1 -- 172.21.15.47:0/961090664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1748004300 con 0x7f174c109900 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.746+0000 7f1750851700 1 -- 172.21.15.47:0/961090664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1748011af0 con 0x7f174c109900 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 -- 172.21.15.47:0/961090664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 msgr2=0x7f174c10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 --2- 172.21.15.47:0/961090664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c10bcf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f17480098a0 tx=0x7f1748009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 -- 172.21.15.47:0/961090664 shutdown_connections 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 --2- 172.21.15.47:0/961090664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c10bcf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 -- 172.21.15.47:0/961090664 >> 172.21.15.47:0/961090664 conn(0x7f174c1032d0 msgr2=0x7f174c105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 -- 172.21.15.47:0/961090664 shutdown_connections 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 -- 172.21.15.47:0/961090664 wait complete. 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 Processor -- start 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.747+0000 7f1753ab7700 1 -- start start 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1753ab7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c109180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1753ab7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f174c10d1a0 con 0x7f174c109900 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1751853700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.538 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1751853700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34238/0 (socket says 172.21.15.47:34238) 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1751853700 1 -- 172.21.15.47:0/2736549592 learned_addr learned my addr 172.21.15.47:0/2736549592 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1751853700 1 -- 172.21.15.47:0/2736549592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c1096c0 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1751853700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c109180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1748003f40 tx=0x7f1748011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17480038b0 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1748003a50 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c1078b0 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f174801a5c0 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.748+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c107df0 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.749+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f1748024430 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.749+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c064bb0 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.750+0000 7f1742ffd700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f173803e360 0x7f1738040820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.750+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1748053880 con 0x7f174c109900 2024-05-11T12:13:16.539 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.750+0000 7f1751052700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f173803e360 0x7f1738040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.750+0000 7f1751052700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f173803e360 0x7f1738040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f173c006fd0 tx=0x7f173c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.755+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f174800b3e0 con 0x7f174c109900 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.760+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f174802d3f0 con 0x7f174c109900 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.985+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f174c1044f0 con 0x7f173803e360 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.990+0000 7f1742ffd700 1 -- 172.21.15.47:0/2736549592 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f174c1044f0 con 0x7f173803e360 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f173803e360 msgr2=0x7f1738040820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f173803e360 0x7f1738040820 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f173c006fd0 tx=0x7f173c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 msgr2=0x7f174c109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c109180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1748003f40 tx=0x7f1748011890 comp rx=0 tx=0).stop 2024-05-11T12:13:16.540 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 shutdown_connections 2024-05-11T12:13:16.541 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f173803e360 0x7f1738040820 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.541 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 --2- 172.21.15.47:0/2736549592 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f174c109900 0x7f174c109180 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:16.541 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 >> 172.21.15.47:0/2736549592 conn(0x7f174c1032d0 msgr2=0x7f174c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:16.541 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 shutdown_connections 2024-05-11T12:13:16.541 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:15.992+0000 7f1753ab7700 1 -- 172.21.15.47:0/2736549592 wait complete. 2024-05-11T12:13:16.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:16 smithi047 ceph-mon[30375]: mgrmap e8: x(active, since 2s) 2024-05-11T12:13:16.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:16 smithi047 ceph-mon[30375]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:16.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:16 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:16.872 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:16 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:17.850 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:17 smithi047 ceph-mon[30375]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:17.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-11T12:13:17.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.984+0000 7f710dc46700 1 Processor -- start 2024-05-11T12:13:17.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.984+0000 7f710dc46700 1 -- start start 2024-05-11T12:13:17.859 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f710dc46700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f71081080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f710dc46700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f71081086b0 con 0x7f7108107cc0 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f71077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f71081080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f71077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f71081080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60664/0 (socket says 172.21.15.47:60664) 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f71077fe700 1 -- 172.21.15.47:0/777518648 learned_addr learned my addr 172.21.15.47:0/777518648 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f71077fe700 1 -- 172.21.15.47:0/777518648 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7108108ec0 con 0x7f7108107cc0 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f71077fe700 1 --2- 172.21.15.47:0/777518648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f71081080e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009e80 tx=0x7f70f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f76946e063d3e6f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.985+0000 7f7106ffd700 1 -- 172.21.15.47:0/777518648 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f8017070 con 0x7f7108107cc0 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f7106ffd700 1 -- 172.21.15.47:0/777518648 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f70f800b8c0 con 0x7f7108107cc0 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f7106ffd700 1 -- 172.21.15.47:0/777518648 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f8004490 con 0x7f7108107cc0 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f710dc46700 1 -- 172.21.15.47:0/777518648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 msgr2=0x7f71081080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f710dc46700 1 --2- 172.21.15.47:0/777518648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f71081080e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009e80 tx=0x7f70f800b330 comp rx=0 tx=0).stop 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f710dc46700 1 -- 172.21.15.47:0/777518648 shutdown_connections 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f710dc46700 1 --2- 172.21.15.47:0/777518648 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f71081080e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:17.860 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f710dc46700 1 -- 172.21.15.47:0/777518648 >> 172.21.15.47:0/777518648 conn(0x7f7108103240 msgr2=0x7f71081056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f710dc46700 1 -- 172.21.15.47:0/777518648 shutdown_connections 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.986+0000 7f710dc46700 1 -- 172.21.15.47:0/777518648 wait complete. 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.987+0000 7f710dc46700 1 Processor -- start 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.987+0000 7f710dc46700 1 -- start start 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.987+0000 7f710dc46700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f7108192e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.987+0000 7f710dc46700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f710810c270 con 0x7f7108107cc0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.987+0000 7f71077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f7108192e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.987+0000 7f71077fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f7108192e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60668/0 (socket says 172.21.15.47:60668) 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.987+0000 7f71077fe700 1 -- 172.21.15.47:0/50414290 learned_addr learned my addr 172.21.15.47:0/50414290 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.988+0000 7f71077fe700 1 -- 172.21.15.47:0/50414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7108193370 con 0x7f7108107cc0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.988+0000 7f71077fe700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f7108192e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70f800b7b0 tx=0x7f70f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.988+0000 7f71057fa700 1 -- 172.21.15.47:0/50414290 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f8020020 con 0x7f7108107cc0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.988+0000 7f71057fa700 1 -- 172.21.15.47:0/50414290 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f70f8012430 con 0x7f7108107cc0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.988+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7108191560 con 0x7f7108107cc0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.988+0000 7f71057fa700 1 -- 172.21.15.47:0/50414290 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f8017040 con 0x7f7108107cc0 2024-05-11T12:13:17.861 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.988+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7108191aa0 con 0x7f7108107cc0 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.989+0000 7f71057fa700 1 -- 172.21.15.47:0/50414290 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f70f8023020 con 0x7f7108107cc0 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.989+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710806a640 con 0x7f7108107cc0 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.989+0000 7f71057fa700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f70f403e470 0x7f70f4040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.990+0000 7f71057fa700 1 -- 172.21.15.47:0/50414290 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70f800f050 con 0x7f7108107cc0 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.990+0000 7f70feffd700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f70f403e470 0x7f70f4040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.990+0000 7f70feffd700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f70f403e470 0x7f70f4040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70f0006fd0 tx=0x7f70f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:16.995+0000 7f71057fa700 1 -- 172.21.15.47:0/50414290 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70f802a020 con 0x7f7108107cc0 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.215+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f7108196c40 con 0x7f70f403e470 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.216+0000 7f71057fa700 1 -- 172.21.15.47:0/50414290 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f7108196c40 con 0x7f70f403e470 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.218+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f70f403e470 msgr2=0x7f70f4040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.218+0000 7f710dc46700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f70f403e470 0x7f70f4040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f70f0006fd0 tx=0x7f70f0006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.218+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 msgr2=0x7f7108192e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:17.862 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.218+0000 7f710dc46700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f7108192e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70f800b7b0 tx=0x7f70f800be20 comp rx=0 tx=0).stop 2024-05-11T12:13:17.863 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.219+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 shutdown_connections 2024-05-11T12:13:17.863 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.219+0000 7f710dc46700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f70f403e470 0x7f70f4040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:17.863 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.219+0000 7f710dc46700 1 --2- 172.21.15.47:0/50414290 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7108107cc0 0x7f7108192e30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:17.863 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.219+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 >> 172.21.15.47:0/50414290 conn(0x7f7108103240 msgr2=0x7f7108104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:17.863 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.219+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 shutdown_connections 2024-05-11T12:13:17.863 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:17.219+0000 7f710dc46700 1 -- 172.21.15.47:0/50414290 wait complete. 2024-05-11T12:13:17.863 INFO:teuthology.orchestra.run.smithi047.stdout:Generating ssh key... 2024-05-11T12:13:19.456 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: Generating public/private rsa key pair. 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: Your identification has been saved in /tmp/tmp6qjy78y0/key. 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: Your public key has been saved in /tmp/tmp6qjy78y0/key.pub. 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: The key fingerprint is: 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: SHA256:GUT/BeoKqwuQ1RaV6mD2AubWW1vlajMquqVC2uigUag ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: The key's randomart image is: 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: +---[RSA 3072]----+ 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: | ...oo . | 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: | . ... . . . | 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: | . o. . o . | 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: |.=+.. = . . | 2024-05-11T12:13:19.457 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: |*+o+ . S . . | 2024-05-11T12:13:19.458 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: |.*..o. + o | 2024-05-11T12:13:19.458 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: |Eo.oo + o | 2024-05-11T12:13:19.458 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: |*.++ o = | 2024-05-11T12:13:19.458 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: |+=o +oo o | 2024-05-11T12:13:19.458 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: +----[SHA256]-----+ 2024-05-11T12:13:19.707 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.309+0000 7fe34e2df700 1 Processor -- start 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.309+0000 7fe34e2df700 1 -- start start 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe34e2df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe34810bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe34e2df700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe34807d690 con 0x7fe348109870 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe347fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe34810bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe347fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe34810bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60684/0 (socket says 172.21.15.47:60684) 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe347fff700 1 -- 172.21.15.47:0/2296832464 learned_addr learned my addr 172.21.15.47:0/2296832464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe347fff700 1 -- 172.21.15.47:0/2296832464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34810c1a0 con 0x7fe348109870 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe347fff700 1 --2- 172.21.15.47:0/2296832464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe34810bc60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098a0 tx=0x7fe338009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4563e1a72b58fd0c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe346ffd700 1 -- 172.21.15.47:0/2296832464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338004160 con 0x7fe348109870 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe346ffd700 1 -- 172.21.15.47:0/2296832464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe338004300 con 0x7fe348109870 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.310+0000 7fe346ffd700 1 -- 172.21.15.47:0/2296832464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338011af0 con 0x7fe348109870 2024-05-11T12:13:19.708 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.311+0000 7fe34e2df700 1 -- 172.21.15.47:0/2296832464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 msgr2=0x7fe34810bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.311+0000 7fe34e2df700 1 --2- 172.21.15.47:0/2296832464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe34810bc60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe3380098a0 tx=0x7fe338009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.311+0000 7fe34e2df700 1 -- 172.21.15.47:0/2296832464 shutdown_connections 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.311+0000 7fe34e2df700 1 --2- 172.21.15.47:0/2296832464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe34810bc60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.311+0000 7fe34e2df700 1 -- 172.21.15.47:0/2296832464 >> 172.21.15.47:0/2296832464 conn(0x7fe348103240 msgr2=0x7fe3481056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.311+0000 7fe34e2df700 1 -- 172.21.15.47:0/2296832464 shutdown_connections 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.311+0000 7fe34e2df700 1 -- 172.21.15.47:0/2296832464 wait complete. 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.312+0000 7fe34e2df700 1 Processor -- start 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.312+0000 7fe34e2df700 1 -- start start 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.312+0000 7fe34e2df700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe3481090f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.312+0000 7fe34e2df700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe34810d160 con 0x7fe348109870 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.312+0000 7fe347fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe3481090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.312+0000 7fe347fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe3481090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60700/0 (socket says 172.21.15.47:60700) 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.312+0000 7fe347fff700 1 -- 172.21.15.47:0/3107971901 learned_addr learned my addr 172.21.15.47:0/3107971901 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:19.709 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.313+0000 7fe347fff700 1 -- 172.21.15.47:0/3107971901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe348109630 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.313+0000 7fe347fff700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe3481090f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe338011ac0 tx=0x7fe338011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.313+0000 7fe3457fa700 1 -- 172.21.15.47:0/3107971901 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338003b20 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.313+0000 7fe3457fa700 1 -- 172.21.15.47:0/3107971901 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe33801a8c0 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.313+0000 7fe3457fa700 1 -- 172.21.15.47:0/3107971901 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3380248d0 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.313+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe348107820 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.313+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe348107d60 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.314+0000 7fe3457fa700 1 -- 172.21.15.47:0/3107971901 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe338022020 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.314+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe348064bb0 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.314+0000 7fe3457fa700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe33003e4c0 0x7fe330040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.314+0000 7fe3457fa700 1 -- 172.21.15.47:0/3107971901 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe3380537b0 con 0x7fe348109870 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.314+0000 7fe3477fe700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe33003e4c0 0x7fe330040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.315+0000 7fe3477fe700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe33003e4c0 0x7fe330040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe33c006fd0 tx=0x7fe33c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:19.710 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.319+0000 7fe3457fa700 1 -- 172.21.15.47:0/3107971901 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe338016df0 con 0x7fe348109870 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:18.539+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fe348104460 con 0x7fe33003e4c0 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.163+0000 7fe3457fa700 1 -- 172.21.15.47:0/3107971901 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe348104460 con 0x7fe33003e4c0 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.165+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe33003e4c0 msgr2=0x7fe330040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.165+0000 7fe34e2df700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe33003e4c0 0x7fe330040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe33c006fd0 tx=0x7fe33c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.165+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 msgr2=0x7fe3481090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.165+0000 7fe34e2df700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe3481090f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe338011ac0 tx=0x7fe338011ba0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.166+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 shutdown_connections 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.166+0000 7fe34e2df700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe33003e4c0 0x7fe330040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.166+0000 7fe34e2df700 1 --2- 172.21.15.47:0/3107971901 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe348109870 0x7fe3481090f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.166+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 >> 172.21.15.47:0/3107971901 conn(0x7fe348103240 msgr2=0x7fe348103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.166+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 shutdown_connections 2024-05-11T12:13:19.711 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:19.166+0000 7fe34e2df700 1 -- 172.21.15.47:0/3107971901 wait complete. 2024-05-11T12:13:20.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:20 smithi047 ceph-mon[30375]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:20.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:20 smithi047 ceph-mon[30375]: Generating ssh key... 2024-05-11T12:13:20.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:20 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:20.369 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:20 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:20.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaoCd0m+U8pU302VFrg3QAol3licF+IbG7+jqn8YtPac6gk43EmmRQdE0O1pqr+8HBUYSDNo0NWtguuw5Oxm4OXh0ns5IOqC3SvbnRBJegMDzeGLfHMOjdrwwXLxs6Ky+9/X3YN4WxHIGBnP5M1Eb955PMUDQSzHViZLnqqzw65UUWoQO8oEGaJpnsyvp7gb1KBonKhF3nCm469rQiHnQCymI9hkiqhoax1kp+pxC+h773oLvIn42wy+80AugbAY3Sx0aulMXsAW3bzczWorX2Wf10gzIOQfPW7AL0LgfaS+7VtEiuQ1UJeZWnMI5zwLXKNXHkstywb8K+oJHEzHB5J1YWjYCqDUSL5BZVyknl5kVuV0oRcUkiIrCNXsWDHUxtbkTzBqI0WiKepR3izEeD9prMcP2057XDwiUe8l0TNxZNmQdvqsxGr+Fd1wmUyEwHf91ae7Cc9wqfIzul8cR6NWhDW76j99bOg1uqx3nVsr7tOinTFuDXJZQ3jS7c20= ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:13:20.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.142+0000 7f2f6738c700 1 Processor -- start 2024-05-11T12:13:20.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.142+0000 7f2f6738c700 1 -- start start 2024-05-11T12:13:20.946 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.143+0000 7f2f6738c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f601080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.143+0000 7f2f6738c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2f601086b0 con 0x7f2f60107cc0 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.143+0000 7f2f65128700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f601080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.143+0000 7f2f65128700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f601080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60712/0 (socket says 172.21.15.47:60712) 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.143+0000 7f2f65128700 1 -- 172.21.15.47:0/2648040479 learned_addr learned my addr 172.21.15.47:0/2648040479 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.143+0000 7f2f65128700 1 -- 172.21.15.47:0/2648040479 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f60108ec0 con 0x7f2f60107cc0 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.143+0000 7f2f65128700 1 --2- 172.21.15.47:0/2648040479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f601080e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bb40 tx=0x7f2f5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cd5ffd2f7dd76a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f53fff700 1 -- 172.21.15.47:0/2648040479 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f5c00dbe0 con 0x7f2f60107cc0 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f53fff700 1 -- 172.21.15.47:0/2648040479 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2f5c0044c0 con 0x7f2f60107cc0 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f53fff700 1 -- 172.21.15.47:0/2648040479 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f5c010470 con 0x7f2f60107cc0 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f6738c700 1 -- 172.21.15.47:0/2648040479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 msgr2=0x7f2f601080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f6738c700 1 --2- 172.21.15.47:0/2648040479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f601080e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bb40 tx=0x7f2f5c00be50 comp rx=0 tx=0).stop 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f6738c700 1 -- 172.21.15.47:0/2648040479 shutdown_connections 2024-05-11T12:13:20.947 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f6738c700 1 --2- 172.21.15.47:0/2648040479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f601080e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f6738c700 1 -- 172.21.15.47:0/2648040479 >> 172.21.15.47:0/2648040479 conn(0x7f2f60103240 msgr2=0x7f2f601056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f6738c700 1 -- 172.21.15.47:0/2648040479 shutdown_connections 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.144+0000 7f2f6738c700 1 -- 172.21.15.47:0/2648040479 wait complete. 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.145+0000 7f2f6738c700 1 Processor -- start 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.145+0000 7f2f6738c700 1 -- start start 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.145+0000 7f2f6738c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f6019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.145+0000 7f2f6738c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6007c7f0 con 0x7f2f60107cc0 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.145+0000 7f2f65128700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f6019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.145+0000 7f2f65128700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f6019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60720/0 (socket says 172.21.15.47:60720) 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.145+0000 7f2f65128700 1 -- 172.21.15.47:0/3740509800 learned_addr learned my addr 172.21.15.47:0/3740509800 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.146+0000 7f2f65128700 1 -- 172.21.15.47:0/3740509800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6019f3f0 con 0x7f2f60107cc0 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.146+0000 7f2f65128700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f6019eeb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bfd0 tx=0x7f2f5c003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.948 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.146+0000 7f2f527fc700 1 -- 172.21.15.47:0/3740509800 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f5c010770 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.146+0000 7f2f527fc700 1 -- 172.21.15.47:0/3740509800 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2f5c01d070 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.146+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6019f680 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.146+0000 7f2f527fc700 1 -- 172.21.15.47:0/3740509800 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f5c019c70 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.146+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601a2370 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.147+0000 7f2f527fc700 1 -- 172.21.15.47:0/3740509800 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2f5c028060 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.147+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f60064bb0 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.147+0000 7f2f527fc700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f2f4c03e420 0x7f2f4c0408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.147+0000 7f2f527fc700 1 -- 172.21.15.47:0/3740509800 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2f5c053940 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.147+0000 7f2f64927700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f2f4c03e420 0x7f2f4c0408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.148+0000 7f2f64927700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f2f4c03e420 0x7f2f4c0408e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006fd0 tx=0x7f2f54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.152+0000 7f2f527fc700 1 -- 172.21.15.47:0/3740509800 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f5c015070 con 0x7f2f60107cc0 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.368+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2f60063990 con 0x7f2f4c03e420 2024-05-11T12:13:20.949 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.369+0000 7f2f527fc700 1 -- 172.21.15.47:0/3740509800 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2f60063990 con 0x7f2f4c03e420 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f2f4c03e420 msgr2=0x7f2f4c0408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f2f4c03e420 0x7f2f4c0408e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f54006fd0 tx=0x7f2f54006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 msgr2=0x7f2f6019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f6019eeb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00bfd0 tx=0x7f2f5c003ff0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 shutdown_connections 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f2f4c03e420 0x7f2f4c0408e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 --2- 172.21.15.47:0/3740509800 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2f60107cc0 0x7f2f6019eeb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 >> 172.21.15.47:0/3740509800 conn(0x7f2f60103240 msgr2=0x7f2f601910e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 shutdown_connections 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:20.371+0000 7f2f6738c700 1 -- 172.21.15.47:0/3740509800 wait complete. 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:Adding key to root@localhost authorized_keys... 2024-05-11T12:13:20.950 INFO:teuthology.orchestra.run.smithi047.stdout:Adding host smithi047... 2024-05-11T12:13:21.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:21 smithi047 ceph-mon[30375]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:22.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:22 smithi047 ceph-mon[30375]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:22.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:22 smithi047 ceph-mon[30375]: Deploying cephadm binary to smithi047 2024-05-11T12:13:26.445 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Added host 'smithi047' with addr '172.21.15.47' 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.333+0000 7fbe2908c700 1 Processor -- start 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.333+0000 7fbe2908c700 1 -- start start 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.333+0000 7fbe2908c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe24109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.333+0000 7fbe2908c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe24109790 con 0x7fbe24108e30 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.333+0000 7fbe22d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe24109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.334+0000 7fbe22d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe24109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60728/0 (socket says 172.21.15.47:60728) 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.334+0000 7fbe22d9d700 1 -- 172.21.15.47:0/4135665305 learned_addr learned my addr 172.21.15.47:0/4135665305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.334+0000 7fbe22d9d700 1 -- 172.21.15.47:0/4135665305 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe24109f90 con 0x7fbe24108e30 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.334+0000 7fbe22d9d700 1 --2- 172.21.15.47:0/4135665305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe24109250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe140098a0 tx=0x7fbe14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfced69c099f94cd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.334+0000 7fbe2259c700 1 -- 172.21.15.47:0/4135665305 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe14004160 con 0x7fbe24108e30 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.334+0000 7fbe2259c700 1 -- 172.21.15.47:0/4135665305 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe14004300 con 0x7fbe24108e30 2024-05-11T12:13:26.446 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.334+0000 7fbe2259c700 1 -- 172.21.15.47:0/4135665305 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe14011af0 con 0x7fbe24108e30 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 -- 172.21.15.47:0/4135665305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 msgr2=0x7fbe24109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 --2- 172.21.15.47:0/4135665305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe24109250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbe140098a0 tx=0x7fbe14009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 -- 172.21.15.47:0/4135665305 shutdown_connections 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 --2- 172.21.15.47:0/4135665305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe24109250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 -- 172.21.15.47:0/4135665305 >> 172.21.15.47:0/4135665305 conn(0x7fbe241043f0 msgr2=0x7fbe24106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 -- 172.21.15.47:0/4135665305 shutdown_connections 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 -- 172.21.15.47:0/4135665305 wait complete. 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.335+0000 7fbe2908c700 1 Processor -- start 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe2908c700 1 -- start start 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe2908c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe2419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe2908c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2410d3d0 con 0x7fbe24108e30 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe22d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe2419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe22d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe2419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60744/0 (socket says 172.21.15.47:60744) 2024-05-11T12:13:26.447 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe22d9d700 1 -- 172.21.15.47:0/495000960 learned_addr learned my addr 172.21.15.47:0/495000960 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe22d9d700 1 -- 172.21.15.47:0/495000960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2419d260 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.336+0000 7fbe22d9d700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe2419cd20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe14003f40 tx=0x7fbe14011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.337+0000 7fbe20d99700 1 -- 172.21.15.47:0/495000960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe140038b0 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.337+0000 7fbe20d99700 1 -- 172.21.15.47:0/495000960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbe14003a50 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.337+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2419d4f0 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.337+0000 7fbe20d99700 1 -- 172.21.15.47:0/495000960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe1401a5c0 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.337+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe241a01d0 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.338+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe24064bb0 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.338+0000 7fbe20d99700 1 -- 172.21.15.47:0/495000960 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fbe1401a870 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.338+0000 7fbe20d99700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fbe1003e470 0x7fbe10040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.338+0000 7fbe20d99700 1 -- 172.21.15.47:0/495000960 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbe140539e0 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.338+0000 7fbe1a59c700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fbe1003e470 0x7fbe10040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.338+0000 7fbe1a59c700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fbe1003e470 0x7fbe10040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006fd0 tx=0x7fbe0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.343+0000 7fbe20d99700 1 -- 172.21.15.47:0/495000960 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe1400e260 con 0x7fbe24108e30 2024-05-11T12:13:26.448 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:21.563+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi047", "addr": "172.21.15.47", "target": ["mon-mgr", ""]}) -- 0x7fbe241055c0 con 0x7fbe1003e470 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.838+0000 7fbe20d99700 1 -- 172.21.15.47:0/495000960 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fbe241055c0 con 0x7fbe1003e470 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fbe1003e470 msgr2=0x7fbe10040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fbe1003e470 0x7fbe10040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006fd0 tx=0x7fbe0c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 msgr2=0x7fbe2419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe2419cd20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbe14003f40 tx=0x7fbe14011890 comp rx=0 tx=0).stop 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 shutdown_connections 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fbe1003e470 0x7fbe10040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 --2- 172.21.15.47:0/495000960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe24108e30 0x7fbe2419cd20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 >> 172.21.15.47:0/495000960 conn(0x7fbe241043f0 msgr2=0x7fbe24104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 shutdown_connections 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:25.841+0000 7fbe2908c700 1 -- 172.21.15.47:0/495000960 wait complete. 2024-05-11T12:13:26.449 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mon service... 2024-05-11T12:13:27.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:26 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:27.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:26 smithi047 ceph-mon[30375]: Added host smithi047 2024-05-11T12:13:27.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:26 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:27.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-11T12:13:27.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f267a3b0700 1 Processor -- start 2024-05-11T12:13:27.846 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f267a3b0700 1 -- start start 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f267a3b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f26741096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f267a3b0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2674109c90 con 0x7f26741092a0 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f2673fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f26741096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f2673fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f26741096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53322/0 (socket says 172.21.15.47:53322) 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f2673fff700 1 -- 172.21.15.47:0/742213879 learned_addr learned my addr 172.21.15.47:0/742213879 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.864+0000 7f2673fff700 1 -- 172.21.15.47:0/742213879 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2674107eb0 con 0x7f26741092a0 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.865+0000 7f2673fff700 1 --2- 172.21.15.47:0/742213879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f26741096c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26640098a0 tx=0x7f2664009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a9bb2e1fdaf6de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.865+0000 7f26737fe700 1 -- 172.21.15.47:0/742213879 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664004160 con 0x7f26741092a0 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.865+0000 7f26737fe700 1 -- 172.21.15.47:0/742213879 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2664004300 con 0x7f26741092a0 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.865+0000 7f26737fe700 1 -- 172.21.15.47:0/742213879 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664011af0 con 0x7f26741092a0 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.866+0000 7f267a3b0700 1 -- 172.21.15.47:0/742213879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 msgr2=0x7f26741096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.866+0000 7f267a3b0700 1 --2- 172.21.15.47:0/742213879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f26741096c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f26640098a0 tx=0x7f2664009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.847 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.866+0000 7f267a3b0700 1 -- 172.21.15.47:0/742213879 shutdown_connections 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.866+0000 7f267a3b0700 1 --2- 172.21.15.47:0/742213879 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f26741096c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.866+0000 7f267a3b0700 1 -- 172.21.15.47:0/742213879 >> 172.21.15.47:0/742213879 conn(0x7f26741029c0 msgr2=0x7f2674104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.866+0000 7f267a3b0700 1 -- 172.21.15.47:0/742213879 shutdown_connections 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.866+0000 7f267a3b0700 1 -- 172.21.15.47:0/742213879 wait complete. 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f267a3b0700 1 Processor -- start 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f267a3b0700 1 -- start start 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f267a3b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f2674192e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f267a3b0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f267410c270 con 0x7f26741092a0 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f2673fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f2674192e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f2673fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f2674192e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53328/0 (socket says 172.21.15.47:53328) 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f2673fff700 1 -- 172.21.15.47:0/2853707160 learned_addr learned my addr 172.21.15.47:0/2853707160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f2673fff700 1 -- 172.21.15.47:0/2853707160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2674193370 con 0x7f26741092a0 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.867+0000 7f2673fff700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f2674192e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26640117b0 tx=0x7f2664011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.848 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.868+0000 7f2671ffb700 1 -- 172.21.15.47:0/2853707160 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664003af0 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.868+0000 7f2671ffb700 1 -- 172.21.15.47:0/2853707160 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2664020070 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.868+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2674191560 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.868+0000 7f2671ffb700 1 -- 172.21.15.47:0/2853707160 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2664019630 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.868+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2674191aa0 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.869+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2674064bb0 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.869+0000 7f2671ffb700 1 -- 172.21.15.47:0/2853707160 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f26640198f0 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.869+0000 7f2671ffb700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f266003e470 0x7f2660040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.869+0000 7f2671ffb700 1 -- 172.21.15.47:0/2853707160 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2664053c40 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.869+0000 7f266bfff700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f266003e470 0x7f2660040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.870+0000 7f266bfff700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f266003e470 0x7f2660040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f265c006fd0 tx=0x7f265c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:26.875+0000 7f2671ffb700 1 -- 172.21.15.47:0/2853707160 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f266400b4d0 con 0x7f26741092a0 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.109+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2674196c40 con 0x7f266003e470 2024-05-11T12:13:27.849 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.114+0000 7f2671ffb700 1 -- 172.21.15.47:0/2853707160 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2674196c40 con 0x7f266003e470 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f266003e470 msgr2=0x7f2660040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f266003e470 0x7f2660040930 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f265c006fd0 tx=0x7f265c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 msgr2=0x7f2674192e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f2674192e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f26640117b0 tx=0x7f2664011b70 comp rx=0 tx=0).stop 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 shutdown_connections 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f266003e470 0x7f2660040930 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 --2- 172.21.15.47:0/2853707160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26741092a0 0x7f2674192e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 >> 172.21.15.47:0/2853707160 conn(0x7f26741029c0 msgr2=0x7f26741038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 shutdown_connections 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:27.116+0000 7f267a3b0700 1 -- 172.21.15.47:0/2853707160 wait complete. 2024-05-11T12:13:27.850 INFO:teuthology.orchestra.run.smithi047.stdout:Deploying unmanaged mgr service... 2024-05-11T12:13:28.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:28 smithi047 ceph-mon[30375]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:28.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:28 smithi047 ceph-mon[30375]: Saving service mon spec with placement count:5 2024-05-11T12:13:28.458 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:28 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:29.128 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.271+0000 7facb13d8700 1 Processor -- start 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.271+0000 7facb13d8700 1 -- start start 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.271+0000 7facb13d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.271+0000 7facb13d8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7facac1097e0 con 0x7facac108df0 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53332/0 (socket says 172.21.15.47:53332) 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaaffd700 1 -- 172.21.15.47:0/299139767 learned_addr learned my addr 172.21.15.47:0/299139767 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaaffd700 1 -- 172.21.15.47:0/299139767 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac109ff0 con 0x7facac108df0 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaaffd700 1 --2- 172.21.15.47:0/299139767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac109210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009e80 tx=0x7fac9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7b9b6b7a46b75c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaa7fc700 1 -- 172.21.15.47:0/299139767 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac9c017070 con 0x7facac108df0 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaa7fc700 1 -- 172.21.15.47:0/299139767 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac9c00b8c0 con 0x7facac108df0 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.272+0000 7facaa7fc700 1 -- 172.21.15.47:0/299139767 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac9c004490 con 0x7facac108df0 2024-05-11T12:13:29.129 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.273+0000 7facb13d8700 1 -- 172.21.15.47:0/299139767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 msgr2=0x7facac109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.273+0000 7facb13d8700 1 --2- 172.21.15.47:0/299139767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac109210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009e80 tx=0x7fac9c00b330 comp rx=0 tx=0).stop 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.273+0000 7facb13d8700 1 -- 172.21.15.47:0/299139767 shutdown_connections 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.273+0000 7facb13d8700 1 --2- 172.21.15.47:0/299139767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac109210 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.273+0000 7facb13d8700 1 -- 172.21.15.47:0/299139767 >> 172.21.15.47:0/299139767 conn(0x7facac1043b0 msgr2=0x7facac1067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.273+0000 7facb13d8700 1 -- 172.21.15.47:0/299139767 shutdown_connections 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.273+0000 7facb13d8700 1 -- 172.21.15.47:0/299139767 wait complete. 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.274+0000 7facb13d8700 1 Processor -- start 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.274+0000 7facb13d8700 1 -- start start 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.274+0000 7facb13d8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac07f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.274+0000 7facb13d8700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7facac10d3a0 con 0x7facac108df0 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.274+0000 7facaaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac07f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.274+0000 7facaaffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac07f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53348/0 (socket says 172.21.15.47:53348) 2024-05-11T12:13:29.130 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.274+0000 7facaaffd700 1 -- 172.21.15.47:0/1056025598 learned_addr learned my addr 172.21.15.47:0/1056025598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.275+0000 7facaaffd700 1 -- 172.21.15.47:0/1056025598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facac07f8d0 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.275+0000 7facaaffd700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac07f390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fac9c004a60 tx=0x7fac9c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.275+0000 7faca8ff9700 1 -- 172.21.15.47:0/1056025598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac9c020020 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.275+0000 7faca8ff9700 1 -- 172.21.15.47:0/1056025598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fac9c003c60 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.275+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facac07dac0 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.275+0000 7faca8ff9700 1 -- 172.21.15.47:0/1056025598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac9c017040 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.275+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facac07e000 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.276+0000 7faca8ff9700 1 -- 172.21.15.47:0/1056025598 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fac9c023020 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.276+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facac064bb0 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.276+0000 7faca8ff9700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fac9803e470 0x7fac98040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.276+0000 7faca8ff9700 1 -- 172.21.15.47:0/1056025598 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fac9c00f050 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.276+0000 7faca27fc700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fac9803e470 0x7fac98040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.277+0000 7faca27fc700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fac9803e470 0x7fac98040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.281+0000 7faca8ff9700 1 -- 172.21.15.47:0/1056025598 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac9c02a020 con 0x7facac108df0 2024-05-11T12:13:29.131 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.521+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7facac1055d0 con 0x7fac9803e470 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.525+0000 7faca8ff9700 1 -- 172.21.15.47:0/1056025598 <== mgr.14120 v2:172.21.15.47:6800/3174346748 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7facac1055d0 con 0x7fac9803e470 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fac9803e470 msgr2=0x7fac98040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fac9803e470 0x7fac98040930 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac94006fd0 tx=0x7fac94006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 msgr2=0x7facac07f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac07f390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fac9c004a60 tx=0x7fac9c00bd50 comp rx=0 tx=0).stop 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 shutdown_connections 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fac9803e470 0x7fac98040930 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 --2- 172.21.15.47:0/1056025598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7facac108df0 0x7facac07f390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 >> 172.21.15.47:0/1056025598 conn(0x7facac1043b0 msgr2=0x7facac104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 shutdown_connections 2024-05-11T12:13:29.132 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:28.528+0000 7facb13d8700 1 -- 172.21.15.47:0/1056025598 wait complete. 2024-05-11T12:13:29.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:29 smithi047 ceph-mon[30375]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:29.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:29 smithi047 ceph-mon[30375]: Saving service mgr spec with placement count:2 2024-05-11T12:13:29.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:29 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:31.064 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.276+0000 7fe8e2d83700 1 Processor -- start 2024-05-11T12:13:31.064 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.276+0000 7fe8e2d83700 1 -- start start 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.276+0000 7fe8e2d83700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc10bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.276+0000 7fe8e2d83700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc07cde0 con 0x7fe8dc109890 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8e0b1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc10bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8e0b1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc10bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53364/0 (socket says 172.21.15.47:53364) 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8e0b1f700 1 -- 172.21.15.47:0/2787235335 learned_addr learned my addr 172.21.15.47:0/2787235335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8e0b1f700 1 -- 172.21.15.47:0/2787235335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc10c1c0 con 0x7fe8dc109890 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8e0b1f700 1 --2- 172.21.15.47:0/2787235335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc10bc80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00098a0 tx=0x7fe8d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa32e4553dc71c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8db7fe700 1 -- 172.21.15.47:0/2787235335 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d0004160 con 0x7fe8dc109890 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8db7fe700 1 -- 172.21.15.47:0/2787235335 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d0004300 con 0x7fe8dc109890 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.277+0000 7fe8db7fe700 1 -- 172.21.15.47:0/2787235335 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d0011a20 con 0x7fe8dc109890 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.278+0000 7fe8e2d83700 1 -- 172.21.15.47:0/2787235335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 msgr2=0x7fe8dc10bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.278+0000 7fe8e2d83700 1 --2- 172.21.15.47:0/2787235335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc10bc80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00098a0 tx=0x7fe8d0009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.278+0000 7fe8e2d83700 1 -- 172.21.15.47:0/2787235335 shutdown_connections 2024-05-11T12:13:31.065 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.278+0000 7fe8e2d83700 1 --2- 172.21.15.47:0/2787235335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc10bc80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.278+0000 7fe8e2d83700 1 -- 172.21.15.47:0/2787235335 >> 172.21.15.47:0/2787235335 conn(0x7fe8dc103240 msgr2=0x7fe8dc1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.278+0000 7fe8e2d83700 1 -- 172.21.15.47:0/2787235335 shutdown_connections 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.278+0000 7fe8e2d83700 1 -- 172.21.15.47:0/2787235335 wait complete. 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e2d83700 1 Processor -- start 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e2d83700 1 -- start start 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e2d83700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e2d83700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe8dc10d170 con 0x7fe8dc109890 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e0b1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e0b1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53368/0 (socket says 172.21.15.47:53368) 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e0b1f700 1 -- 172.21.15.47:0/148692644 learned_addr learned my addr 172.21.15.47:0/148692644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.279+0000 7fe8e0b1f700 1 -- 172.21.15.47:0/148692644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8dc19d240 con 0x7fe8dc109890 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.280+0000 7fe8e0b1f700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc19cd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0004660 tx=0x7fe8d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.280+0000 7fe8d9ffb700 1 -- 172.21.15.47:0/148692644 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d0003aa0 con 0x7fe8dc109890 2024-05-11T12:13:31.066 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.280+0000 7fe8d9ffb700 1 -- 172.21.15.47:0/148692644 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8d001b8f0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.280+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8dc19d4d0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.280+0000 7fe8d9ffb700 1 -- 172.21.15.47:0/148692644 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d0023900 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.280+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8dc1a01b0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.281+0000 7fe8d9ffb700 1 -- 172.21.15.47:0/148692644 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe8d0023bc0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.281+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8dc064bb0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.281+0000 7fe8d9ffb700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe8cc03e470 0x7fe8cc040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.281+0000 7fe8d9ffb700 1 -- 172.21.15.47:0/148692644 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe8d0053eb0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.281+0000 7fe8dbfff700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe8cc03e470 0x7fe8cc040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.282+0000 7fe8dbfff700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe8cc03e470 0x7fe8cc040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006fd0 tx=0x7fe8c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.286+0000 7fe8d9ffb700 1 -- 172.21.15.47:0/148692644 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe8d001f030 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.502+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe8dc1a07a0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.504+0000 7fe8d9ffb700 1 -- 172.21.15.47:0/148692644 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe8d00292a0 con 0x7fe8dc109890 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe8cc03e470 msgr2=0x7fe8cc040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe8cc03e470 0x7fe8cc040930 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8006fd0 tx=0x7fe8c8006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:31.067 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 msgr2=0x7fe8dc19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:31.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc19cd00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0004660 tx=0x7fe8d0011aa0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 shutdown_connections 2024-05-11T12:13:31.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7fe8cc03e470 0x7fe8cc040930 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 --2- 172.21.15.47:0/148692644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8dc109890 0x7fe8dc19cd00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:31.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 >> 172.21.15.47:0/148692644 conn(0x7fe8dc103240 msgr2=0x7fe8dc103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:31.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 shutdown_connections 2024-05-11T12:13:31.068 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:30.507+0000 7fe8e2d83700 1 -- 172.21.15.47:0/148692644 wait complete. 2024-05-11T12:13:31.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:31 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:31.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:31 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/148692644' entity='client.admin' 2024-05-11T12:13:31.459 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:31 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:32.490 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.472+0000 7f65d4888700 1 Processor -- start 2024-05-11T12:13:32.491 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.472+0000 7f65d4888700 1 -- start start 2024-05-11T12:13:32.491 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.472+0000 7f65d4888700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d0109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.491 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.472+0000 7f65d4888700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65d0109790 con 0x7f65d0108e30 2024-05-11T12:13:32.491 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.472+0000 7f65ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d0109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.491 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.472+0000 7f65ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d0109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53384/0 (socket says 172.21.15.47:53384) 2024-05-11T12:13:32.491 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.472+0000 7f65ce59c700 1 -- 172.21.15.47:0/1718285883 learned_addr learned my addr 172.21.15.47:0/1718285883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:32.491 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.473+0000 7f65ce59c700 1 -- 172.21.15.47:0/1718285883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d0109f90 con 0x7f65d0108e30 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.473+0000 7f65ce59c700 1 --2- 172.21.15.47:0/1718285883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d0109250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65c00098a0 tx=0x7f65c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8871b8a2225c9c54 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.473+0000 7f65cd59a700 1 -- 172.21.15.47:0/1718285883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0004160 con 0x7f65d0108e30 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.473+0000 7f65cd59a700 1 -- 172.21.15.47:0/1718285883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c0004300 con 0x7f65d0108e30 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.473+0000 7f65cd59a700 1 -- 172.21.15.47:0/1718285883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0011a20 con 0x7f65d0108e30 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.474+0000 7f65d4888700 1 -- 172.21.15.47:0/1718285883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 msgr2=0x7f65d0109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.474+0000 7f65d4888700 1 --2- 172.21.15.47:0/1718285883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d0109250 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65c00098a0 tx=0x7f65c0009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.474+0000 7f65d4888700 1 -- 172.21.15.47:0/1718285883 shutdown_connections 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.474+0000 7f65d4888700 1 --2- 172.21.15.47:0/1718285883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d0109250 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.474+0000 7f65d4888700 1 -- 172.21.15.47:0/1718285883 >> 172.21.15.47:0/1718285883 conn(0x7f65d01043f0 msgr2=0x7f65d0106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.474+0000 7f65d4888700 1 -- 172.21.15.47:0/1718285883 shutdown_connections 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.474+0000 7f65d4888700 1 -- 172.21.15.47:0/1718285883 wait complete. 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65d4888700 1 Processor -- start 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65d4888700 1 -- start start 2024-05-11T12:13:32.492 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65d4888700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65d4888700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65d010d3d0 con 0x7f65d0108e30 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53396/0 (socket says 172.21.15.47:53396) 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65ce59c700 1 -- 172.21.15.47:0/2591658792 learned_addr learned my addr 172.21.15.47:0/2591658792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65ce59c700 1 -- 172.21.15.47:0/2591658792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d019d1e0 con 0x7f65d0108e30 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.475+0000 7f65ce59c700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d019cca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65c00116e0 tx=0x7f65c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.476+0000 7f65bf7fe700 1 -- 172.21.15.47:0/2591658792 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0003a60 con 0x7f65d0108e30 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.476+0000 7f65bf7fe700 1 -- 172.21.15.47:0/2591658792 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c001b8f0 con 0x7f65d0108e30 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.476+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d019d470 con 0x7f65d0108e30 2024-05-11T12:13:32.493 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.476+0000 7f65bf7fe700 1 -- 172.21.15.47:0/2591658792 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65c0019860 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.476+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d01a0160 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.477+0000 7f65bf7fe700 1 -- 172.21.15.47:0/2591658792 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f65c0019b10 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.477+0000 7f65bf7fe700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f65b803e470 0x7f65b8040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.477+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d010d550 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.477+0000 7f65bf7fe700 1 -- 172.21.15.47:0/2591658792 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f65c0053c30 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.477+0000 7f65cdd9b700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f65b803e470 0x7f65b8040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.477+0000 7f65cdd9b700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f65b803e470 0x7f65b8040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006fd0 tx=0x7f65c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.482+0000 7f65bf7fe700 1 -- 172.21.15.47:0/2591658792 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65c001f030 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.713+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f65d0064bb0 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.715+0000 7f65bf7fe700 1 -- 172.21.15.47:0/2591658792 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f65c001bda0 con 0x7f65d0108e30 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f65b803e470 msgr2=0x7f65b8040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.494 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f65b803e470 0x7f65b8040930 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f65c4006fd0 tx=0x7f65c4006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 msgr2=0x7f65d019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d019cca0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f65c00116e0 tx=0x7f65c0011aa0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 shutdown_connections 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f65b803e470 0x7f65b8040930 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 --2- 172.21.15.47:0/2591658792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65d0108e30 0x7f65d019cca0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 >> 172.21.15.47:0/2591658792 conn(0x7f65d01043f0 msgr2=0x7f65d0104db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 shutdown_connections 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:31.718+0000 7f65d4888700 1 -- 172.21.15.47:0/2591658792 wait complete. 2024-05-11T12:13:32.495 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling the dashboard module... 2024-05-11T12:13:33.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:32 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2591658792' entity='client.admin' 2024-05-11T12:13:34.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:33 smithi047 ceph-mon[30375]: from='mgr.14120 172.21.15.47:0/983770348' entity='mgr.x' 2024-05-11T12:13:34.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:33 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/230520522' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-11T12:13:34.178 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: ignoring --setuser ceph since I am not root 2024-05-11T12:13:34.178 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: ignoring --setgroup ceph since I am not root 2024-05-11T12:13:34.178 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:34.114+0000 7fb7ee838200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T12:13:34.524 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:34.317+0000 7fb7ee838200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:13:34.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:34.423+0000 7fb7ee838200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f4e482700 1 Processor -- start 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f4e482700 1 -- start start 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f4e482700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f481080e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f4e482700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7f481086b0 con 0x7f7f48107cc0 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f481080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f481080e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53404/0 (socket says 172.21.15.47:53404) 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f47fff700 1 -- 172.21.15.47:0/1719177918 learned_addr learned my addr 172.21.15.47:0/1719177918 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.250+0000 7f7f47fff700 1 -- 172.21.15.47:0/1719177918 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48108ec0 con 0x7f7f48107cc0 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.251+0000 7f7f47fff700 1 --2- 172.21.15.47:0/1719177918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f481080e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396ddc4e517e9866 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:34.547 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.251+0000 7f7f46ffd700 1 -- 172.21.15.47:0/1719177918 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38004160 con 0x7f7f48107cc0 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.251+0000 7f7f46ffd700 1 -- 172.21.15.47:0/1719177918 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f38004300 con 0x7f7f48107cc0 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.251+0000 7f7f46ffd700 1 -- 172.21.15.47:0/1719177918 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f38011a20 con 0x7f7f48107cc0 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.251+0000 7f7f4e482700 1 -- 172.21.15.47:0/1719177918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 msgr2=0x7f7f481080e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 --2- 172.21.15.47:0/1719177918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f481080e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7f380098a0 tx=0x7f7f38009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 -- 172.21.15.47:0/1719177918 shutdown_connections 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 --2- 172.21.15.47:0/1719177918 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f481080e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 -- 172.21.15.47:0/1719177918 >> 172.21.15.47:0/1719177918 conn(0x7f7f48103240 msgr2=0x7f7f481056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 -- 172.21.15.47:0/1719177918 shutdown_connections 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 -- 172.21.15.47:0/1719177918 wait complete. 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 Processor -- start 2024-05-11T12:13:34.548 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.252+0000 7f7f4e482700 1 -- start start 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f4e482700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f48192e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f4e482700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4810c270 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f48192e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f48192e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53410/0 (socket says 172.21.15.47:53410) 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f47fff700 1 -- 172.21.15.47:0/230520522 learned_addr learned my addr 172.21.15.47:0/230520522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f47fff700 1 -- 172.21.15.47:0/230520522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48193370 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f47fff700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f48192e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f380119f0 tx=0x7f7f38011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f380039e0 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f3801b8f0 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f48191560 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.253+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f380237a0 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.254+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f48191aa0 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.254+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f7f38023aa0 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.255+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f48064bb0 con 0x7f7f48107cc0 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.255+0000 7f7f457fa700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7f3003e420 0x7f7f300408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:34.549 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.255+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7f38053dc0 con 0x7f7f48107cc0 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.255+0000 7f7f477fe700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7f3003e420 0x7f7f300408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.255+0000 7f7f477fe700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7f3003e420 0x7f7f300408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006fd0 tx=0x7f7f3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.260+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7f3801f030 con 0x7f7f48107cc0 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.543+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f7f48196b20 con 0x7f7f48107cc0 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.922+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f7f3800b3e0 con 0x7f7f48107cc0 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.924+0000 7f7f457fa700 1 -- 172.21.15.47:0/230520522 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f7f3802c3f0 con 0x7f7f48107cc0 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7f3003e420 msgr2=0x7f7f300408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7f3003e420 0x7f7f300408e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c006fd0 tx=0x7f7f3c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:34.550 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 msgr2=0x7f7f48192e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:34.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f48192e30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f380119f0 tx=0x7f7f38011ad0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 shutdown_connections 2024-05-11T12:13:34.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f7f3003e420 0x7f7f300408e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 --2- 172.21.15.47:0/230520522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7f48107cc0 0x7f7f48192e30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:34.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.926+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 >> 172.21.15.47:0/230520522 conn(0x7f7f48103240 msgr2=0x7f7f48104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:34.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.927+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 shutdown_connections 2024-05-11T12:13:34.551 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:33.927+0000 7f7f4e482700 1 -- 172.21.15.47:0/230520522 wait complete. 2024-05-11T12:13:34.789 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:34.754+0000 7fb7ee838200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T12:13:35.042 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:34.846+0000 7fb7ee838200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T12:13:35.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:34 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/230520522' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-11T12:13:35.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:34 smithi047 ceph-mon[30375]: mgrmap e9: x(active, since 20s) 2024-05-11T12:13:35.962 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:13:35.962 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-11T12:13:35.962 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-11T12:13:35.963 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-11T12:13:35.963 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-11T12:13:35.963 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:13:35.963 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.031+0000 7f67cc0b9700 1 Processor -- start 2024-05-11T12:13:35.963 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.031+0000 7f67cc0b9700 1 -- start start 2024-05-11T12:13:35.963 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.031+0000 7f67cc0b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c410bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.964 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.031+0000 7f67cc0b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f67c407d670 con 0x7f67c4109860 2024-05-11T12:13:35.964 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.031+0000 7f67c9e55700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c410bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.964 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.031+0000 7f67c9e55700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c410bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53446/0 (socket says 172.21.15.47:53446) 2024-05-11T12:13:35.964 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.031+0000 7f67c9e55700 1 -- 172.21.15.47:0/1301857552 learned_addr learned my addr 172.21.15.47:0/1301857552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:35.964 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.032+0000 7f67c9e55700 1 -- 172.21.15.47:0/1301857552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c410c190 con 0x7f67c4109860 2024-05-11T12:13:35.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.032+0000 7f67c9e55700 1 --2- 172.21.15.47:0/1301857552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c410bc50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67c00098a0 tx=0x7f67c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd04dd65c0b652a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:35.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.032+0000 7f67c8e53700 1 -- 172.21.15.47:0/1301857552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c0004160 con 0x7f67c4109860 2024-05-11T12:13:35.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.032+0000 7f67c8e53700 1 -- 172.21.15.47:0/1301857552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c0004300 con 0x7f67c4109860 2024-05-11T12:13:35.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.032+0000 7f67c8e53700 1 -- 172.21.15.47:0/1301857552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c0011a20 con 0x7f67c4109860 2024-05-11T12:13:35.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1301857552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 msgr2=0x7f67c410bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.965 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 --2- 172.21.15.47:0/1301857552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c410bc50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f67c00098a0 tx=0x7f67c0009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1301857552 shutdown_connections 2024-05-11T12:13:35.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 --2- 172.21.15.47:0/1301857552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c410bc50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1301857552 >> 172.21.15.47:0/1301857552 conn(0x7f67c4103250 msgr2=0x7f67c4105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:35.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1301857552 shutdown_connections 2024-05-11T12:13:35.966 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1301857552 wait complete. 2024-05-11T12:13:35.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.033+0000 7f67cc0b9700 1 Processor -- start 2024-05-11T12:13:35.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67cc0b9700 1 -- start start 2024-05-11T12:13:35.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67cc0b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c41090e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67cc0b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f67c410d140 con 0x7f67c4109860 2024-05-11T12:13:35.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67c9e55700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c41090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:35.967 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67c9e55700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c41090e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53452/0 (socket says 172.21.15.47:53452) 2024-05-11T12:13:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67c9e55700 1 -- 172.21.15.47:0/1019934487 learned_addr learned my addr 172.21.15.47:0/1019934487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67c9e55700 1 -- 172.21.15.47:0/1019934487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c4109620 con 0x7f67c4109860 2024-05-11T12:13:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67c9e55700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c41090e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67c00116e0 tx=0x7f67c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.034+0000 7f67baffd700 1 -- 172.21.15.47:0/1019934487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c0003a60 con 0x7f67c4109860 2024-05-11T12:13:35.968 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.035+0000 7f67baffd700 1 -- 172.21.15.47:0/1019934487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c001b8f0 con 0x7f67c4109860 2024-05-11T12:13:35.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.035+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c4107810 con 0x7f67c4109860 2024-05-11T12:13:35.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.035+0000 7f67baffd700 1 -- 172.21.15.47:0/1019934487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67c0019860 con 0x7f67c4109860 2024-05-11T12:13:35.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.035+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c4107d50 con 0x7f67c4109860 2024-05-11T12:13:35.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.036+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c4064bb0 con 0x7f67c4109860 2024-05-11T12:13:35.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.036+0000 7f67baffd700 1 -- 172.21.15.47:0/1019934487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f67c0019b60 con 0x7f67c4109860 2024-05-11T12:13:35.969 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.036+0000 7f67baffd700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 0x7f67b0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:35.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.036+0000 7f67c9654700 1 -- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 msgr2=0x7f67b0040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3174346748 2024-05-11T12:13:35.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.036+0000 7f67baffd700 1 -- 172.21.15.47:0/1019934487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f67c0053cb0 con 0x7f67c4109860 2024-05-11T12:13:35.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.036+0000 7f67c9654700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 0x7f67b0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:13:35.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.041+0000 7f67baffd700 1 -- 172.21.15.47:0/1019934487 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f67c001f030 con 0x7f67c4109860 2024-05-11T12:13:35.970 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.237+0000 7f67c9654700 1 -- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 msgr2=0x7f67b0040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3174346748 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.237+0000 7f67c9654700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 0x7f67b0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.360+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f67c4108090 con 0x7f67c4109860 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.360+0000 7f67baffd700 1 -- 172.21.15.47:0/1019934487 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f67c4108090 con 0x7f67c4109860 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 msgr2=0x7f67b0040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:13:35.971 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 0x7f67b0040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 msgr2=0x7f67c41090e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:35.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c41090e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67c00116e0 tx=0x7f67c0011aa0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 shutdown_connections 2024-05-11T12:13:35.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f67b003e470 0x7f67b0040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.972 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 --2- 172.21.15.47:0/1019934487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67c4109860 0x7f67c41090e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:35.973 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 >> 172.21.15.47:0/1019934487 conn(0x7f67c4103250 msgr2=0x7f67c4103cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:35.973 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 shutdown_connections 2024-05-11T12:13:35.973 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:35.365+0000 7f67cc0b9700 1 -- 172.21.15.47:0/1019934487 wait complete. 2024-05-11T12:13:35.973 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for the mgr to restart... 2024-05-11T12:13:35.974 INFO:teuthology.orchestra.run.smithi047.stdout:Waiting for mgr epoch 9... 2024-05-11T12:13:36.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:35 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1019934487' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-11T12:13:36.185 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:36.145+0000 7fb7ee838200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T12:13:36.508 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:36.327+0000 7fb7ee838200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T12:13:36.508 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:36.508+0000 7fb7ee838200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T12:13:37.121 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:36.812+0000 7fb7ee838200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T12:13:37.121 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:36.909+0000 7fb7ee838200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T12:13:37.121 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:37.121+0000 7fb7ee838200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T12:13:37.524 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:37.323+0000 7fb7ee838200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:13:38.188 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:37.956+0000 7fb7ee838200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T12:13:38.188 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:38.188+0000 7fb7ee838200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T12:13:39.024 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:38.634+0000 7fb7ee838200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T12:13:39.025 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:38.736+0000 7fb7ee838200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T12:13:39.406 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:39.124+0000 7fb7ee838200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T12:13:39.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:39.500+0000 7fb7ee838200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T12:13:39.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:39.597+0000 7fb7ee838200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T12:13:40.275 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:39.857+0000 7fb7ee838200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T12:13:40.275 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:39.999+0000 7fb7ee838200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T12:13:40.275 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:40.100+0000 7fb7ee838200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T12:13:40.774 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:40.462+0000 7fb7ee838200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T12:13:40.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:40.560+0000 7fb7ee838200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T12:13:41.480 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:41.197+0000 7fb7ee838200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T12:13:41.481 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:13:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:13:41.427+0000 7fb7ee838200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: Active manager daemon x restarted 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: Activating manager daemon x 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: osdmap e3: 0 total, 0 up, 0 in 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: mgrmap e10: x(active, starting, since 0.00372749s) 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-11T12:13:41.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-11T12:13:41.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: Manager daemon x is now available 2024-05-11T12:13:41.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:41.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-11T12:13:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-11T12:13:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:42 smithi047 ceph-mon[30375]: [11/May/2024:12:13:41] ENGINE Bus STARTING 2024-05-11T12:13:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:42 smithi047 ceph-mon[30375]: [11/May/2024:12:13:41] ENGINE Serving on http://172.21.15.47:8765 2024-05-11T12:13:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:42 smithi047 ceph-mon[30375]: [11/May/2024:12:13:41] ENGINE Serving on https://172.21.15.47:7150 2024-05-11T12:13:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:42 smithi047 ceph-mon[30375]: [11/May/2024:12:13:41] ENGINE Bus STARTED 2024-05-11T12:13:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:42 smithi047 ceph-mon[30375]: mgrmap e11: x(active, since 1.00634s) 2024-05-11T12:13:43.265 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout { 2024-05-11T12:13:43.273 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-11T12:13:43.273 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout } 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.397+0000 7f4263054700 1 Processor -- start 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.397+0000 7f4263054700 1 -- start start 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.397+0000 7f4263054700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.397+0000 7f4263054700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f425c07ce00 con 0x7f425c107660 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f4260df0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f4260df0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53458/0 (socket says 172.21.15.47:53458) 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f4260df0700 1 -- 172.21.15.47:0/2060892606 learned_addr learned my addr 172.21.15.47:0/2060892606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f4260df0700 1 -- 172.21.15.47:0/2060892606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c107fc0 con 0x7f425c107660 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f4260df0700 1 --2- 172.21.15.47:0/2060892606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c107a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42500098a0 tx=0x7f4250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99ae9bbcc653ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:43.274 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f425b7fe700 1 -- 172.21.15.47:0/2060892606 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4250004160 con 0x7f425c107660 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f425b7fe700 1 -- 172.21.15.47:0/2060892606 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4250004300 con 0x7f425c107660 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.398+0000 7f425b7fe700 1 -- 172.21.15.47:0/2060892606 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4250011a20 con 0x7f425c107660 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.399+0000 7f4263054700 1 -- 172.21.15.47:0/2060892606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 msgr2=0x7f425c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.399+0000 7f4263054700 1 --2- 172.21.15.47:0/2060892606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c107a80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f42500098a0 tx=0x7f4250009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.399+0000 7f4263054700 1 -- 172.21.15.47:0/2060892606 shutdown_connections 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.399+0000 7f4263054700 1 --2- 172.21.15.47:0/2060892606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c107a80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.399+0000 7f4263054700 1 -- 172.21.15.47:0/2060892606 >> 172.21.15.47:0/2060892606 conn(0x7f425c103240 msgr2=0x7f425c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.399+0000 7f4263054700 1 -- 172.21.15.47:0/2060892606 shutdown_connections 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.399+0000 7f4263054700 1 -- 172.21.15.47:0/2060892606 wait complete. 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.400+0000 7f4263054700 1 Processor -- start 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.400+0000 7f4263054700 1 -- start start 2024-05-11T12:13:43.275 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.400+0000 7f4263054700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.400+0000 7f4263054700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f425c109030 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.400+0000 7f4260df0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.400+0000 7f4260df0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53468/0 (socket says 172.21.15.47:53468) 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.400+0000 7f4260df0700 1 -- 172.21.15.47:0/117641357 learned_addr learned my addr 172.21.15.47:0/117641357 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.401+0000 7f4260df0700 1 -- 172.21.15.47:0/117641357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c19f450 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.401+0000 7f4260df0700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c19ef10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42500116e0 tx=0x7f4250011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.401+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4250003b20 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.401+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c19f6e0 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.401+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f425001b8f0 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.401+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42500238c0 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.401+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c1a23d0 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.402+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4250023b70 con 0x7f425c107660 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.402+0000 7f4259ffb700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:43.276 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.402+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 --> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f424c041040 con 0x7f424c03e470 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.402+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4250054000 con 0x7f425c107660 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.402+0000 7f425bfff700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 msgr2=0x7f424c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3174346748 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.402+0000 7f425bfff700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.603+0000 7f425bfff700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 msgr2=0x7f424c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3174346748 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:36.603+0000 7f425bfff700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:37.004+0000 7f425bfff700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 msgr2=0x7f424c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3174346748 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:37.004+0000 7f425bfff700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:37.805+0000 7f425bfff700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 msgr2=0x7f424c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3174346748 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:37.805+0000 7f425bfff700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:39.406+0000 7f425bfff700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 msgr2=0x7f424c040930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.47:6800/3174346748 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:39.406+0000 7f425bfff700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-11T12:13:43.277 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:41.431+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f425001baa0 con 0x7f425c107660 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:41.431+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 msgr2=0x7f424c040930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:41.431+0000 7f4259ffb700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.433+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f4250055c60 con 0x7f425c107660 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.433+0000 7f4259ffb700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f424c041ef0 0x7f424c0442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.433+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4250008e50 con 0x7f424c041ef0 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.434+0000 7f425bfff700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f424c041ef0 0x7f424c0442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.434+0000 7f425bfff700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f424c041ef0 0x7f424c0442e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4248003a10 tx=0x7f42480092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.435+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4250008e50 con 0x7f424c041ef0 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.443+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f425c06a640 con 0x7f424c041ef0 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4259ffb700 1 -- 172.21.15.47:0/117641357 <== mgr.14152 v2:172.21.15.47:6800/811043456 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f425c06a640 con 0x7f424c041ef0 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f424c041ef0 msgr2=0x7f424c0442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:43.278 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f424c041ef0 0x7f424c0442e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4248003a10 tx=0x7f42480092b0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 msgr2=0x7f425c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c19ef10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42500116e0 tx=0x7f4250011aa0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 shutdown_connections 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f424c041ef0 0x7f424c0442e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f425c107660 0x7f425c19ef10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 --2- 172.21.15.47:0/117641357 >> [v2:172.21.15.47:6800/3174346748,v1:172.21.15.47:6801/3174346748] conn(0x7f424c03e470 0x7f424c040930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 >> 172.21.15.47:0/117641357 conn(0x7f425c103240 msgr2=0x7f425c195b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 shutdown_connections 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:42.444+0000 7f4263054700 1 -- 172.21.15.47:0/117641357 wait complete. 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:mgr epoch 9 is available 2024-05-11T12:13:43.279 INFO:teuthology.orchestra.run.smithi047.stdout:Generating a dashboard self-signed certificate... 2024-05-11T12:13:43.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:43 smithi047 ceph-mon[30375]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-11T12:13:43.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:43 smithi047 ceph-mon[30375]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-11T12:13:44.644 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:44 smithi047 ceph-mon[30375]: mgrmap e12: x(active, since 2s) 2024-05-11T12:13:44.644 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:44 smithi047 ceph-mon[30375]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:44.644 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:44.645 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:44.645 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:44.652 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-11T12:13:44.652 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.683+0000 7f1e799b6700 1 Processor -- start 2024-05-11T12:13:44.652 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.683+0000 7f1e799b6700 1 -- start start 2024-05-11T12:13:44.652 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.683+0000 7f1e799b6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7407db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.683+0000 7f1e799b6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7407e0f0 con 0x7f1e7407f6c0 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e72ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7407db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e72ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7407db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53018/0 (socket says 172.21.15.47:53018) 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e72ffd700 1 -- 172.21.15.47:0/3253199609 learned_addr learned my addr 172.21.15.47:0/3253199609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e72ffd700 1 -- 172.21.15.47:0/3253199609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7407e270 con 0x7f1e7407f6c0 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e72ffd700 1 --2- 172.21.15.47:0/3253199609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7407db20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009e80 tx=0x7f1e6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548fb507fb2ec9a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e71ffb700 1 -- 172.21.15.47:0/3253199609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e64017070 con 0x7f1e7407f6c0 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e71ffb700 1 -- 172.21.15.47:0/3253199609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e6400b8c0 con 0x7f1e7407f6c0 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.684+0000 7f1e71ffb700 1 -- 172.21.15.47:0/3253199609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e64004490 con 0x7f1e7407f6c0 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 -- 172.21.15.47:0/3253199609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 msgr2=0x7f1e7407db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 --2- 172.21.15.47:0/3253199609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7407db20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009e80 tx=0x7f1e6400b330 comp rx=0 tx=0).stop 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 -- 172.21.15.47:0/3253199609 shutdown_connections 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 --2- 172.21.15.47:0/3253199609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7407db20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 -- 172.21.15.47:0/3253199609 >> 172.21.15.47:0/3253199609 conn(0x7f1e741043f0 msgr2=0x7f1e74106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:44.653 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 -- 172.21.15.47:0/3253199609 shutdown_connections 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 -- 172.21.15.47:0/3253199609 wait complete. 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.685+0000 7f1e799b6700 1 Processor -- start 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e799b6700 1 -- start start 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e799b6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e799b6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7410f430 con 0x7f1e7407f6c0 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e72ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e72ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53020/0 (socket says 172.21.15.47:53020) 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e72ffd700 1 -- 172.21.15.47:0/3079413954 learned_addr learned my addr 172.21.15.47:0/3079413954 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e72ffd700 1 -- 172.21.15.47:0/3079413954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7419f4a0 con 0x7f1e7407f6c0 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.686+0000 7f1e72ffd700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7419ef60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400b7b0 tx=0x7f1e6400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.687+0000 7f1e789b4700 1 -- 172.21.15.47:0/3079413954 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e64020020 con 0x7f1e7407f6c0 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.687+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7419f730 con 0x7f1e7407f6c0 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.687+0000 7f1e789b4700 1 -- 172.21.15.47:0/3079413954 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e64003b50 con 0x7f1e7407f6c0 2024-05-11T12:13:44.654 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.687+0000 7f1e789b4700 1 -- 172.21.15.47:0/3079413954 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e64017040 con 0x7f1e7407f6c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.687+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e741a2420 con 0x7f1e7407f6c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.688+0000 7f1e789b4700 1 -- 172.21.15.47:0/3079413954 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f1e64023020 con 0x7f1e7407f6c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.688+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e74064bb0 con 0x7f1e7407f6c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.688+0000 7f1e789b4700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e6003e4c0 0x7f1e60040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.688+0000 7f1e789b4700 1 -- 172.21.15.47:0/3079413954 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1e6400f050 con 0x7f1e7407f6c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.688+0000 7f1e727fc700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e6003e4c0 0x7f1e60040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.688+0000 7f1e727fc700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e6003e4c0 0x7f1e60040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c006fd0 tx=0x7f1e5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.693+0000 7f1e789b4700 1 -- 172.21.15.47:0/3079413954 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e6402a020 con 0x7f1e7407f6c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:43.921+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f1e74105610 con 0x7f1e6003e4c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.031+0000 7f1e789b4700 1 -- 172.21.15.47:0/3079413954 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f1e74105610 con 0x7f1e6003e4c0 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e6003e4c0 msgr2=0x7f1e60040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e6003e4c0 0x7f1e60040980 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c006fd0 tx=0x7f1e5c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 msgr2=0x7f1e7419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7419ef60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400b7b0 tx=0x7f1e6400bd30 comp rx=0 tx=0).stop 2024-05-11T12:13:44.655 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 shutdown_connections 2024-05-11T12:13:44.656 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e6003e4c0 0x7f1e60040980 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.656 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 --2- 172.21.15.47:0/3079413954 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e7407f6c0 0x7f1e7419ef60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:44.656 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 >> 172.21.15.47:0/3079413954 conn(0x7f1e741043f0 msgr2=0x7f1e74104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:44.656 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 shutdown_connections 2024-05-11T12:13:44.656 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:44.034+0000 7f1e799b6700 1 -- 172.21.15.47:0/3079413954 wait complete. 2024-05-11T12:13:44.656 INFO:teuthology.orchestra.run.smithi047.stdout:Creating initial admin user... 2024-05-11T12:13:46.217 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8H/DrQqTemaGG/.QdTl5I.Wf8KyHtURJkzJrfVDcQoVM7zC/aos5a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715429625, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-11T12:13:46.217 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.140+0000 7fc5789ed700 1 Processor -- start 2024-05-11T12:13:46.217 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.140+0000 7fc5789ed700 1 -- start start 2024-05-11T12:13:46.217 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.140+0000 7fc5789ed700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc574109210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.140+0000 7fc5789ed700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc5741097e0 con 0x7fc574108df0 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.140+0000 7fc57259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc574109210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.140+0000 7fc57259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc574109210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53034/0 (socket says 172.21.15.47:53034) 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.140+0000 7fc57259c700 1 -- 172.21.15.47:0/1449622710 learned_addr learned my addr 172.21.15.47:0/1449622710 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.141+0000 7fc57259c700 1 -- 172.21.15.47:0/1449622710 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc574109ff0 con 0x7fc574108df0 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.141+0000 7fc57259c700 1 --2- 172.21.15.47:0/1449622710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc574109210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5640098a0 tx=0x7fc564009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6d0b3524420d30 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.141+0000 7fc57159a700 1 -- 172.21.15.47:0/1449622710 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc564004160 con 0x7fc574108df0 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.141+0000 7fc57159a700 1 -- 172.21.15.47:0/1449622710 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc564004300 con 0x7fc574108df0 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.141+0000 7fc57159a700 1 -- 172.21.15.47:0/1449622710 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc564011a20 con 0x7fc574108df0 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.142+0000 7fc5789ed700 1 -- 172.21.15.47:0/1449622710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 msgr2=0x7fc574109210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.142+0000 7fc5789ed700 1 --2- 172.21.15.47:0/1449622710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc574109210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5640098a0 tx=0x7fc564009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.142+0000 7fc5789ed700 1 -- 172.21.15.47:0/1449622710 shutdown_connections 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.142+0000 7fc5789ed700 1 --2- 172.21.15.47:0/1449622710 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc574109210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.218 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.142+0000 7fc5789ed700 1 -- 172.21.15.47:0/1449622710 >> 172.21.15.47:0/1449622710 conn(0x7fc5741043b0 msgr2=0x7fc5741067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.142+0000 7fc5789ed700 1 -- 172.21.15.47:0/1449622710 shutdown_connections 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.142+0000 7fc5789ed700 1 -- 172.21.15.47:0/1449622710 wait complete. 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc5789ed700 1 Processor -- start 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc5789ed700 1 -- start start 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc5789ed700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc57419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc5789ed700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc57410d3a0 con 0x7fc574108df0 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc57259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc57419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc57259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc57419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53050/0 (socket says 172.21.15.47:53050) 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc57259c700 1 -- 172.21.15.47:0/1109620667 learned_addr learned my addr 172.21.15.47:0/1109620667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc57259c700 1 -- 172.21.15.47:0/1109620667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57419d280 con 0x7fc574108df0 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.143+0000 7fc57259c700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc57419cd40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5640116e0 tx=0x7fc564011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.144+0000 7fc56b7fe700 1 -- 172.21.15.47:0/1109620667 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5640039e0 con 0x7fc574108df0 2024-05-11T12:13:46.219 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.144+0000 7fc56b7fe700 1 -- 172.21.15.47:0/1109620667 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56401b8f0 con 0x7fc574108df0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.144+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57419d510 con 0x7fc574108df0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.144+0000 7fc56b7fe700 1 -- 172.21.15.47:0/1109620667 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc564023760 con 0x7fc574108df0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.144+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741a0200 con 0x7fc574108df0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.145+0000 7fc56b7fe700 1 -- 172.21.15.47:0/1109620667 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fc56401b440 con 0x7fc574108df0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.145+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc574064bb0 con 0x7fc574108df0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.145+0000 7fc56b7fe700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc56003e470 0x7fc560040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.145+0000 7fc56b7fe700 1 -- 172.21.15.47:0/1109620667 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc564053f60 con 0x7fc574108df0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.145+0000 7fc571d9b700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc56003e470 0x7fc560040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.145+0000 7fc571d9b700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc56003e470 0x7fc560040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc55c006fd0 tx=0x7fc55c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:46.221 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.151+0000 7fc56b7fe700 1 -- 172.21.15.47:0/1109620667 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc56401f030 con 0x7fc574108df0 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.398+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc5741a03d0 con 0x7fc56003e470 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.644+0000 7fc56b7fe700 1 -- 172.21.15.47:0/1109620667 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc5741a03d0 con 0x7fc56003e470 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc56003e470 msgr2=0x7fc560040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc56003e470 0x7fc560040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc55c006fd0 tx=0x7fc55c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 msgr2=0x7fc57419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc57419cd40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc5640116e0 tx=0x7fc564011aa0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 shutdown_connections 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc56003e470 0x7fc560040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 --2- 172.21.15.47:0/1109620667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc574108df0 0x7fc57419cd40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 >> 172.21.15.47:0/1109620667 conn(0x7fc5741043b0 msgr2=0x7fc574104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 shutdown_connections 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:45.646+0000 7fc5789ed700 1 -- 172.21.15.47:0/1109620667 wait complete. 2024-05-11T12:13:46.222 INFO:teuthology.orchestra.run.smithi047.stdout:Fetching dashboard port number... 2024-05-11T12:13:46.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:46.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:46 smithi047 ceph-mon[30375]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:46.474 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:48.330 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stdout 8443 2024-05-11T12:13:48.330 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197f84e700 1 Processor -- start 2024-05-11T12:13:48.330 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197f84e700 1 -- start start 2024-05-11T12:13:48.330 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197f84e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f1978109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197f84e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1978109790 con 0x7f1978108e30 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197d5ea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f1978109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197d5ea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f1978109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40840/0 (socket says 172.21.15.47:40840) 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197d5ea700 1 -- 172.21.15.47:0/1390077654 learned_addr learned my addr 172.21.15.47:0/1390077654 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.299+0000 7f197d5ea700 1 -- 172.21.15.47:0/1390077654 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1978109f90 con 0x7f1978108e30 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f197d5ea700 1 --2- 172.21.15.47:0/1390077654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f1978109250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e80 tx=0x7f197400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26d7fd69c147bb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f196ffff700 1 -- 172.21.15.47:0/1390077654 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974017070 con 0x7f1978108e30 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f196ffff700 1 -- 172.21.15.47:0/1390077654 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f197400b8c0 con 0x7f1978108e30 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f196ffff700 1 -- 172.21.15.47:0/1390077654 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974004490 con 0x7f1978108e30 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f197f84e700 1 -- 172.21.15.47:0/1390077654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 msgr2=0x7f1978109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f197f84e700 1 --2- 172.21.15.47:0/1390077654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f1978109250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1974009e80 tx=0x7f197400b330 comp rx=0 tx=0).stop 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f197f84e700 1 -- 172.21.15.47:0/1390077654 shutdown_connections 2024-05-11T12:13:48.331 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f197f84e700 1 --2- 172.21.15.47:0/1390077654 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f1978109250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f197f84e700 1 -- 172.21.15.47:0/1390077654 >> 172.21.15.47:0/1390077654 conn(0x7f19781043f0 msgr2=0x7f1978106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.300+0000 7f197f84e700 1 -- 172.21.15.47:0/1390077654 shutdown_connections 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197f84e700 1 -- 172.21.15.47:0/1390077654 wait complete. 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197f84e700 1 Processor -- start 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197f84e700 1 -- start start 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197f84e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f197807f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197f84e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f197810d3d0 con 0x7f1978108e30 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197d5ea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f197807f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197d5ea700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f197807f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40854/0 (socket says 172.21.15.47:40854) 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.301+0000 7f197d5ea700 1 -- 172.21.15.47:0/1798296262 learned_addr learned my addr 172.21.15.47:0/1798296262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.302+0000 7f197d5ea700 1 -- 172.21.15.47:0/1798296262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197807f8d0 con 0x7f1978108e30 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.302+0000 7f197d5ea700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f197807f390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1974004220 tx=0x7f197400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.302+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974017040 con 0x7f1978108e30 2024-05-11T12:13:48.332 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.302+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1974022070 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.302+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197807dac0 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.302+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974012430 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.302+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197807e000 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.303+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f197401d440 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.303+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1978064bb0 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.303+0000 7f196e7fc700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f196403e4c0 0x7f1964040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.303+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f197400f050 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.304+0000 7f197cde9700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f196403e4c0 0x7f1964040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.304+0000 7f197cde9700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f196403e4c0 0x7f1964040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1968006fd0 tx=0x7f1968006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.309+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f197401b3f0 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.484+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1974025360 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.527+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f197807e930 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.528+0000 7f196e7fc700 1 -- 172.21.15.47:0/1798296262 <== mon.0 v2:172.21.15.47:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f197401aa50 con 0x7f1978108e30 2024-05-11T12:13:48.333 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f196403e4c0 msgr2=0x7f1964040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f196403e4c0 0x7f1964040980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1968006fd0 tx=0x7f1968006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 msgr2=0x7f197807f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f197807f390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1974004220 tx=0x7f197400bd30 comp rx=0 tx=0).stop 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 shutdown_connections 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f196403e4c0 0x7f1964040980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 --2- 172.21.15.47:0/1798296262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1978108e30 0x7f197807f390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 >> 172.21.15.47:0/1798296262 conn(0x7f19781043f0 msgr2=0x7f1978104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 shutdown_connections 2024-05-11T12:13:48.334 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:47.530+0000 7f197f84e700 1 -- 172.21.15.47:0/1798296262 wait complete. 2024-05-11T12:13:48.341 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-11T12:13:48.341 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout disabled 2024-05-11T12:13:48.349 INFO:teuthology.orchestra.run.smithi047.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-11T12:13:48.350 INFO:teuthology.orchestra.run.smithi047.stdout:systemctl: stdout inactive 2024-05-11T12:13:48.350 INFO:teuthology.orchestra.run.smithi047.stdout:firewalld.service is not enabled 2024-05-11T12:13:48.350 INFO:teuthology.orchestra.run.smithi047.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-11T12:13:48.352 INFO:teuthology.orchestra.run.smithi047.stdout:Ceph Dashboard is now available at: 2024-05-11T12:13:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: URL: https://smithi047.front.sepia.ceph.com:8443/ 2024-05-11T12:13:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: User: admin 2024-05-11T12:13:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: Password: 8r7ffll0s4 2024-05-11T12:13:48.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:48.353 INFO:teuthology.orchestra.run.smithi047.stdout:Saving cluster configuration to /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config directory 2024-05-11T12:13:48.353 INFO:teuthology.orchestra.run.smithi047.stdout:Enabling autotune for osd_memory_target 2024-05-11T12:13:48.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:48 smithi047 ceph-mon[30375]: mgrmap e13: x(active, since 6s) 2024-05-11T12:13:48.600 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:48 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1798296262' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-11T12:13:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.809+0000 7fe014774700 1 Processor -- start 2024-05-11T12:13:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.809+0000 7fe014774700 1 -- start start 2024-05-11T12:13:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.809+0000 7fe014774700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c10bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.611 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.809+0000 7fe014774700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c07d690 con 0x7fe00c1098d0 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe012510700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c10bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe012510700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c10bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40856/0 (socket says 172.21.15.47:40856) 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe012510700 1 -- 172.21.15.47:0/734775031 learned_addr learned my addr 172.21.15.47:0/734775031 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe012510700 1 -- 172.21.15.47:0/734775031 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c10c200 con 0x7fe00c1098d0 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe012510700 1 --2- 172.21.15.47:0/734775031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c10bcc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe008009e80 tx=0x7fe00800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26a8e21998c04fa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe01150e700 1 -- 172.21.15.47:0/734775031 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008017070 con 0x7fe00c1098d0 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe01150e700 1 -- 172.21.15.47:0/734775031 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00800b8c0 con 0x7fe00c1098d0 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.810+0000 7fe01150e700 1 -- 172.21.15.47:0/734775031 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008004490 con 0x7fe00c1098d0 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.811+0000 7fe014774700 1 -- 172.21.15.47:0/734775031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 msgr2=0x7fe00c10bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.811+0000 7fe014774700 1 --2- 172.21.15.47:0/734775031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c10bcc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe008009e80 tx=0x7fe00800b330 comp rx=0 tx=0).stop 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.811+0000 7fe014774700 1 -- 172.21.15.47:0/734775031 shutdown_connections 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.811+0000 7fe014774700 1 --2- 172.21.15.47:0/734775031 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c10bcc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.811+0000 7fe014774700 1 -- 172.21.15.47:0/734775031 >> 172.21.15.47:0/734775031 conn(0x7fe00c1032e0 msgr2=0x7fe00c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:49.612 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.811+0000 7fe014774700 1 -- 172.21.15.47:0/734775031 shutdown_connections 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.811+0000 7fe014774700 1 -- 172.21.15.47:0/734775031 wait complete. 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe014774700 1 Processor -- start 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe014774700 1 -- start start 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe014774700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe014774700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10d200 con 0x7fe00c1098d0 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe012510700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe012510700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c19cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40870/0 (socket says 172.21.15.47:40870) 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe012510700 1 -- 172.21.15.47:0/609918018 learned_addr learned my addr 172.21.15.47:0/609918018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.812+0000 7fe012510700 1 -- 172.21.15.47:0/609918018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c19d290 con 0x7fe00c1098d0 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.813+0000 7fe012510700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c19cd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe00800b7b0 tx=0x7fe00800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.813+0000 7fe0037fe700 1 -- 172.21.15.47:0/609918018 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008020020 con 0x7fe00c1098d0 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.813+0000 7fe0037fe700 1 -- 172.21.15.47:0/609918018 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe008003c60 con 0x7fe00c1098d0 2024-05-11T12:13:49.613 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.813+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c19d520 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.813+0000 7fe0037fe700 1 -- 172.21.15.47:0/609918018 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe008017040 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.813+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c1a0210 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.814+0000 7fe0037fe700 1 -- 172.21.15.47:0/609918018 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe008023020 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.814+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00c064bb0 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.814+0000 7fe0037fe700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff803e5c0 0x7fdff8040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.814+0000 7fe0037fe700 1 -- 172.21.15.47:0/609918018 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe00800f050 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.815+0000 7fe011d0f700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff803e5c0 0x7fdff8040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.815+0000 7fe011d0f700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff803e5c0 0x7fdff8040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006fd0 tx=0x7fdffc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:48.819+0000 7fe0037fe700 1 -- 172.21.15.47:0/609918018 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe00802a020 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.035+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe00c1a03e0 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.035+0000 7fe0037fe700 1 -- 172.21.15.47:0/609918018 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe00c1a03e0 con 0x7fe00c1098d0 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff803e5c0 msgr2=0x7fdff8040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.614 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff803e5c0 0x7fdff8040a80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdffc006fd0 tx=0x7fdffc006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 msgr2=0x7fe00c19cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c19cd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe00800b7b0 tx=0x7fe00800bd50 comp rx=0 tx=0).stop 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 shutdown_connections 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff803e5c0 0x7fdff8040a80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 --2- 172.21.15.47:0/609918018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c1098d0 0x7fe00c19cd50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 >> 172.21.15.47:0/609918018 conn(0x7fe00c1032e0 msgr2=0x7fe00c103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 shutdown_connections 2024-05-11T12:13:49.615 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:49.038+0000 7fe014774700 1 -- 172.21.15.47:0/609918018 wait complete. 2024-05-11T12:13:51.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.045+0000 7f08f737f700 1 Processor -- start 2024-05-11T12:13:51.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.045+0000 7f08f737f700 1 -- start start 2024-05-11T12:13:51.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.045+0000 7f08f737f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f007db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.045+0000 7f08f737f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08f007e0d0 con 0x7f08f007f6a0 2024-05-11T12:13:51.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.045+0000 7f08f511b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f007db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.045+0000 7f08f511b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f007db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40884/0 (socket says 172.21.15.47:40884) 2024-05-11T12:13:51.001 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.045+0000 7f08f511b700 1 -- 172.21.15.47:0/3949229848 learned_addr learned my addr 172.21.15.47:0/3949229848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.046+0000 7f08f511b700 1 -- 172.21.15.47:0/3949229848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f007e250 con 0x7f08f007f6a0 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.046+0000 7f08f511b700 1 --2- 172.21.15.47:0/3949229848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f007db00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0098a0 tx=0x7f08ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c00a9790eea9869 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.046+0000 7f08e7fff700 1 -- 172.21.15.47:0/3949229848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ec004160 con 0x7f08f007f6a0 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.046+0000 7f08e7fff700 1 -- 172.21.15.47:0/3949229848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ec004300 con 0x7f08f007f6a0 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.046+0000 7f08e7fff700 1 -- 172.21.15.47:0/3949229848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ec011a20 con 0x7f08f007f6a0 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 -- 172.21.15.47:0/3949229848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 msgr2=0x7f08f007db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 --2- 172.21.15.47:0/3949229848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f007db00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0098a0 tx=0x7f08ec009bb0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 -- 172.21.15.47:0/3949229848 shutdown_connections 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 --2- 172.21.15.47:0/3949229848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f007db00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 -- 172.21.15.47:0/3949229848 >> 172.21.15.47:0/3949229848 conn(0x7f08f01043f0 msgr2=0x7f08f0106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 -- 172.21.15.47:0/3949229848 shutdown_connections 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 -- 172.21.15.47:0/3949229848 wait complete. 2024-05-11T12:13:51.002 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.047+0000 7f08f737f700 1 Processor -- start 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f737f700 1 -- start start 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f737f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f737f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f08f010f450 con 0x7f08f007f6a0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f511b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f511b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f019ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40894/0 (socket says 172.21.15.47:40894) 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f511b700 1 -- 172.21.15.47:0/829363051 learned_addr learned my addr 172.21.15.47:0/829363051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f511b700 1 -- 172.21.15.47:0/829363051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f019f3d0 con 0x7f08f007f6a0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08f511b700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f019ee90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08ec000c00 tx=0x7f08ec011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08e67fc700 1 -- 172.21.15.47:0/829363051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ec003aa0 con 0x7f08f007f6a0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.048+0000 7f08e67fc700 1 -- 172.21.15.47:0/829363051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ec01b8f0 con 0x7f08f007f6a0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.049+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f019f660 con 0x7f08f007f6a0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.049+0000 7f08e67fc700 1 -- 172.21.15.47:0/829363051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08ec023870 con 0x7f08f007f6a0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.049+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f01a2350 con 0x7f08f007f6a0 2024-05-11T12:13:51.003 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.049+0000 7f08e67fc700 1 -- 172.21.15.47:0/829363051 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f08ec023b20 con 0x7f08f007f6a0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.050+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f0064bb0 con 0x7f08f007f6a0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.050+0000 7f08e67fc700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08dc03e5c0 0x7f08dc040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.050+0000 7f08e67fc700 1 -- 172.21.15.47:0/829363051 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f08ec053f80 con 0x7f08f007f6a0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.050+0000 7f08f491a700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08dc03e5c0 0x7f08dc040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.050+0000 7f08f491a700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08dc03e5c0 0x7f08dc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006fd0 tx=0x7f08e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.055+0000 7f08e67fc700 1 -- 172.21.15.47:0/829363051 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f08ec01f030 con 0x7f08f007f6a0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.389+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f08f019fb40 con 0x7f08f007f6a0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.391+0000 7f08e67fc700 1 -- 172.21.15.47:0/829363051 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f08f019fb40 con 0x7f08f007f6a0 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08dc03e5c0 msgr2=0x7f08dc040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08dc03e5c0 0x7f08dc040a80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f08e0006fd0 tx=0x7f08e0006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 msgr2=0x7f08f019ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f019ee90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08ec000c00 tx=0x7f08ec011aa0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.004 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 shutdown_connections 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08dc03e5c0 0x7f08dc040a80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 --2- 172.21.15.47:0/829363051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f08f007f6a0 0x7f08f019ee90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.393+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 >> 172.21.15.47:0/829363051 conn(0x7f08f01043f0 msgr2=0x7f08f0104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.394+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 shutdown_connections 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr 2024-05-11T12:13:50.394+0000 7f08f737f700 1 -- 172.21.15.47:0/829363051 wait complete. 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:Or, if you are only running a single cluster on this host: 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.005 INFO:teuthology.orchestra.run.smithi047.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout: ceph telemetry on 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout:For more information see: 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:13:51.006 INFO:teuthology.orchestra.run.smithi047.stdout:Bootstrap complete. 2024-05-11T12:13:51.041 INFO:tasks.cephadm:Fetching config... 2024-05-11T12:13:51.041 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:13:51.041 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-11T12:13:51.058 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-11T12:13:51.058 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:13:51.058 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-11T12:13:51.115 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-11T12:13:51.115 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:13:51.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-11T12:13:51.181 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-11T12:13:51.181 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:13:51.181 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-11T12:13:51.237 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-11T12:13:51.237 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaoCd0m+U8pU302VFrg3QAol3licF+IbG7+jqn8YtPac6gk43EmmRQdE0O1pqr+8HBUYSDNo0NWtguuw5Oxm4OXh0ns5IOqC3SvbnRBJegMDzeGLfHMOjdrwwXLxs6Ky+9/X3YN4WxHIGBnP5M1Eb955PMUDQSzHViZLnqqzw65UUWoQO8oEGaJpnsyvp7gb1KBonKhF3nCm469rQiHnQCymI9hkiqhoax1kp+pxC+h773oLvIn42wy+80AugbAY3Sx0aulMXsAW3bzczWorX2Wf10gzIOQfPW7AL0LgfaS+7VtEiuQ1UJeZWnMI5zwLXKNXHkstywb8K+oJHEzHB5J1YWjYCqDUSL5BZVyknl5kVuV0oRcUkiIrCNXsWDHUxtbkTzBqI0WiKepR3izEeD9prMcP2057XDwiUe8l0TNxZNmQdvqsxGr+Fd1wmUyEwHf91ae7Cc9wqfIzul8cR6NWhDW76j99bOg1uqx3nVsr7tOinTFuDXJZQ3jS7c20= ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-11T12:13:51.320 INFO:teuthology.orchestra.run.smithi047.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaoCd0m+U8pU302VFrg3QAol3licF+IbG7+jqn8YtPac6gk43EmmRQdE0O1pqr+8HBUYSDNo0NWtguuw5Oxm4OXh0ns5IOqC3SvbnRBJegMDzeGLfHMOjdrwwXLxs6Ky+9/X3YN4WxHIGBnP5M1Eb955PMUDQSzHViZLnqqzw65UUWoQO8oEGaJpnsyvp7gb1KBonKhF3nCm469rQiHnQCymI9hkiqhoax1kp+pxC+h773oLvIn42wy+80AugbAY3Sx0aulMXsAW3bzczWorX2Wf10gzIOQfPW7AL0LgfaS+7VtEiuQ1UJeZWnMI5zwLXKNXHkstywb8K+oJHEzHB5J1YWjYCqDUSL5BZVyknl5kVuV0oRcUkiIrCNXsWDHUxtbkTzBqI0WiKepR3izEeD9prMcP2057XDwiUe8l0TNxZNmQdvqsxGr+Fd1wmUyEwHf91ae7Cc9wqfIzul8cR6NWhDW76j99bOg1uqx3nVsr7tOinTFuDXJZQ3jS7c20= ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:13:51.337 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaoCd0m+U8pU302VFrg3QAol3licF+IbG7+jqn8YtPac6gk43EmmRQdE0O1pqr+8HBUYSDNo0NWtguuw5Oxm4OXh0ns5IOqC3SvbnRBJegMDzeGLfHMOjdrwwXLxs6Ky+9/X3YN4WxHIGBnP5M1Eb955PMUDQSzHViZLnqqzw65UUWoQO8oEGaJpnsyvp7gb1KBonKhF3nCm469rQiHnQCymI9hkiqhoax1kp+pxC+h773oLvIn42wy+80AugbAY3Sx0aulMXsAW3bzczWorX2Wf10gzIOQfPW7AL0LgfaS+7VtEiuQ1UJeZWnMI5zwLXKNXHkstywb8K+oJHEzHB5J1YWjYCqDUSL5BZVyknl5kVuV0oRcUkiIrCNXsWDHUxtbkTzBqI0WiKepR3izEeD9prMcP2057XDwiUe8l0TNxZNmQdvqsxGr+Fd1wmUyEwHf91ae7Cc9wqfIzul8cR6NWhDW76j99bOg1uqx3nVsr7tOinTFuDXJZQ3jS7c20= ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-11T12:13:51.377 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaoCd0m+U8pU302VFrg3QAol3licF+IbG7+jqn8YtPac6gk43EmmRQdE0O1pqr+8HBUYSDNo0NWtguuw5Oxm4OXh0ns5IOqC3SvbnRBJegMDzeGLfHMOjdrwwXLxs6Ky+9/X3YN4WxHIGBnP5M1Eb955PMUDQSzHViZLnqqzw65UUWoQO8oEGaJpnsyvp7gb1KBonKhF3nCm469rQiHnQCymI9hkiqhoax1kp+pxC+h773oLvIn42wy+80AugbAY3Sx0aulMXsAW3bzczWorX2Wf10gzIOQfPW7AL0LgfaS+7VtEiuQ1UJeZWnMI5zwLXKNXHkstywb8K+oJHEzHB5J1YWjYCqDUSL5BZVyknl5kVuV0oRcUkiIrCNXsWDHUxtbkTzBqI0WiKepR3izEeD9prMcP2057XDwiUe8l0TNxZNmQdvqsxGr+Fd1wmUyEwHf91ae7Cc9wqfIzul8cR6NWhDW76j99bOg1uqx3nVsr7tOinTFuDXJZQ3jS7c20= ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:13:51.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaoCd0m+U8pU302VFrg3QAol3licF+IbG7+jqn8YtPac6gk43EmmRQdE0O1pqr+8HBUYSDNo0NWtguuw5Oxm4OXh0ns5IOqC3SvbnRBJegMDzeGLfHMOjdrwwXLxs6Ky+9/X3YN4WxHIGBnP5M1Eb955PMUDQSzHViZLnqqzw65UUWoQO8oEGaJpnsyvp7gb1KBonKhF3nCm469rQiHnQCymI9hkiqhoax1kp+pxC+h773oLvIn42wy+80AugbAY3Sx0aulMXsAW3bzczWorX2Wf10gzIOQfPW7AL0LgfaS+7VtEiuQ1UJeZWnMI5zwLXKNXHkstywb8K+oJHEzHB5J1YWjYCqDUSL5BZVyknl5kVuV0oRcUkiIrCNXsWDHUxtbkTzBqI0WiKepR3izEeD9prMcP2057XDwiUe8l0TNxZNmQdvqsxGr+Fd1wmUyEwHf91ae7Cc9wqfIzul8cR6NWhDW76j99bOg1uqx3nVsr7tOinTFuDXJZQ3jS7c20= ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-11T12:13:51.430 INFO:teuthology.orchestra.run.smithi064.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwaoCd0m+U8pU302VFrg3QAol3licF+IbG7+jqn8YtPac6gk43EmmRQdE0O1pqr+8HBUYSDNo0NWtguuw5Oxm4OXh0ns5IOqC3SvbnRBJegMDzeGLfHMOjdrwwXLxs6Ky+9/X3YN4WxHIGBnP5M1Eb955PMUDQSzHViZLnqqzw65UUWoQO8oEGaJpnsyvp7gb1KBonKhF3nCm469rQiHnQCymI9hkiqhoax1kp+pxC+h773oLvIn42wy+80AugbAY3Sx0aulMXsAW3bzczWorX2Wf10gzIOQfPW7AL0LgfaS+7VtEiuQ1UJeZWnMI5zwLXKNXHkstywb8K+oJHEzHB5J1YWjYCqDUSL5BZVyknl5kVuV0oRcUkiIrCNXsWDHUxtbkTzBqI0WiKepR3izEeD9prMcP2057XDwiUe8l0TNxZNmQdvqsxGr+Fd1wmUyEwHf91ae7Cc9wqfIzul8cR6NWhDW76j99bOg1uqx3nVsr7tOinTFuDXJZQ3jS7c20= ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:13:51.443 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-11T12:13:51.698 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:13:51.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:51 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/829363051' entity='client.admin' 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 -- 172.21.15.47:0/2650669663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 msgr2=0x7faa1c105ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 --2- 172.21.15.47:0/2650669663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c105ef0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faa18009910 tx=0x7faa18009c20 comp rx=0 tx=0).stop 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 -- 172.21.15.47:0/2650669663 shutdown_connections 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 --2- 172.21.15.47:0/2650669663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c105ef0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 -- 172.21.15.47:0/2650669663 >> 172.21.15.47:0/2650669663 conn(0x7faa1c0fd450 msgr2=0x7faa1c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 -- 172.21.15.47:0/2650669663 shutdown_connections 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 -- 172.21.15.47:0/2650669663 wait complete. 2024-05-11T12:13:53.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.937+0000 7faa23f44700 1 Processor -- start 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa23f44700 1 -- start start 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa23f44700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa23f44700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c108ef0 con 0x7faa1c103b00 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa21ce0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa21ce0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40910/0 (socket says 172.21.15.47:40910) 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa21ce0700 1 -- 172.21.15.47:0/3600151713 learned_addr learned my addr 172.21.15.47:0/3600151713 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa21ce0700 1 -- 172.21.15.47:0/3600151713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c198ad0 con 0x7faa1c103b00 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa21ce0700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c198590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faa18005b40 tx=0x7faa18004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa12ffd700 1 -- 172.21.15.47:0/3600151713 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa18027070 con 0x7faa1c103b00 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa12ffd700 1 -- 172.21.15.47:0/3600151713 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1800be10 con 0x7faa1c103b00 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c198d60 con 0x7faa1c103b00 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.938+0000 7faa12ffd700 1 -- 172.21.15.47:0/3600151713 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa18022de0 con 0x7faa1c103b00 2024-05-11T12:13:53.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.939+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c19ba50 con 0x7faa1c103b00 2024-05-11T12:13:53.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.939+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1c109670 con 0x7faa1c103b00 2024-05-11T12:13:53.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.940+0000 7faa12ffd700 1 -- 172.21.15.47:0/3600151713 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7faa1802a070 con 0x7faa1c103b00 2024-05-11T12:13:53.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.940+0000 7faa12ffd700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa0803e570 0x7faa08040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:53.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.940+0000 7faa12ffd700 1 -- 172.21.15.47:0/3600151713 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faa1801e030 con 0x7faa1c103b00 2024-05-11T12:13:53.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.940+0000 7faa214df700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa0803e570 0x7faa08040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:53.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.940+0000 7faa214df700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa0803e570 0x7faa08040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa0c006fd0 tx=0x7faa0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:53.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:53.945+0000 7faa12ffd700 1 -- 172.21.15.47:0/3600151713 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faa18013070 con 0x7faa1c103b00 2024-05-11T12:13:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi047", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:13:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:54.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.159+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7faa1c02e680 con 0x7faa1c103b00 2024-05-11T12:13:54.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.161+0000 7faa12ffd700 1 -- 172.21.15.47:0/3600151713 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7faa18031020 con 0x7faa1c103b00 2024-05-11T12:13:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.163+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa0803e570 msgr2=0x7faa08040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.163+0000 7faa23f44700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa0803e570 0x7faa08040a30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa0c006fd0 tx=0x7faa0c006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.163+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 msgr2=0x7faa1c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.163+0000 7faa23f44700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c198590 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faa18005b40 tx=0x7faa18004f80 comp rx=0 tx=0).stop 2024-05-11T12:13:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.164+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 shutdown_connections 2024-05-11T12:13:54.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.164+0000 7faa23f44700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa0803e570 0x7faa08040a30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.164+0000 7faa23f44700 1 --2- 172.21.15.47:0/3600151713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa1c103b00 0x7faa1c198590 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.164+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 >> 172.21.15.47:0/3600151713 conn(0x7faa1c0fd450 msgr2=0x7faa1c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.164+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 shutdown_connections 2024-05-11T12:13:54.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:54.164+0000 7faa23f44700 1 -- 172.21.15.47:0/3600151713 wait complete. 2024-05-11T12:13:54.711 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-11T12:13:54.711 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-11T12:13:54.962 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:13:55.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:55 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3600151713' entity='client.admin' 2024-05-11T12:13:55.336 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:55 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:55.337 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:55 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 -- 172.21.15.47:0/3200170627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 msgr2=0x7f978c1022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 --2- 172.21.15.47:0/3200170627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c1022a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f977c009910 tx=0x7f977c009c20 comp rx=0 tx=0).stop 2024-05-11T12:13:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 -- 172.21.15.47:0/3200170627 shutdown_connections 2024-05-11T12:13:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 --2- 172.21.15.47:0/3200170627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c1022a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 -- 172.21.15.47:0/3200170627 >> 172.21.15.47:0/3200170627 conn(0x7f978c0fd490 msgr2=0x7f978c0ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:56.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 -- 172.21.15.47:0/3200170627 shutdown_connections 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 -- 172.21.15.47:0/3200170627 wait complete. 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 Processor -- start 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.486+0000 7f979234c700 1 -- start start 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f979234c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c0ff300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f979234c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f978c107d80 con 0x7f978c101e80 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f978bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c0ff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f978bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c0ff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40926/0 (socket says 172.21.15.47:40926) 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f978bfff700 1 -- 172.21.15.47:0/2967302616 learned_addr learned my addr 172.21.15.47:0/2967302616 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f978bfff700 1 -- 172.21.15.47:0/2967302616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c0ff840 con 0x7f978c101e80 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f978bfff700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c0ff300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f977c004fb0 tx=0x7f977c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f9789ffb700 1 -- 172.21.15.47:0/2967302616 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f977c027070 con 0x7f978c101e80 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f9789ffb700 1 -- 172.21.15.47:0/2967302616 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f977c00be10 con 0x7f978c101e80 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c0fda30 con 0x7f978c101e80 2024-05-11T12:13:56.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f9789ffb700 1 -- 172.21.15.47:0/2967302616 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f977c022de0 con 0x7f978c101e80 2024-05-11T12:13:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.487+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c0fdf70 con 0x7f978c101e80 2024-05-11T12:13:56.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.488+0000 7f9789ffb700 1 -- 172.21.15.47:0/2967302616 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f977c02a070 con 0x7f978c101e80 2024-05-11T12:13:56.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.488+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c108500 con 0x7f978c101e80 2024-05-11T12:13:56.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.489+0000 7f9789ffb700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f977803e510 0x7f97780409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:56.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.489+0000 7f9789ffb700 1 -- 172.21.15.47:0/2967302616 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f977c01e030 con 0x7f978c101e80 2024-05-11T12:13:56.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.489+0000 7f9783fff700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f977803e510 0x7f97780409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:56.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.489+0000 7f9783fff700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f977803e510 0x7f97780409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9774006fd0 tx=0x7f9774006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:56.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.494+0000 7f9789ffb700 1 -- 172.21.15.47:0/2967302616 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f977c013070 con 0x7f978c101e80 2024-05-11T12:13:56.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.711+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f978c191b70 con 0x7f977803e510 2024-05-11T12:13:56.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.714+0000 7f9789ffb700 1 -- 172.21.15.47:0/2967302616 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f978c191b70 con 0x7f977803e510 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f977803e510 msgr2=0x7f97780409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f977803e510 0x7f97780409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9774006fd0 tx=0x7f9774006e40 comp rx=0 tx=0).stop 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 msgr2=0x7f978c0ff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c0ff300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f977c004fb0 tx=0x7f977c009f60 comp rx=0 tx=0).stop 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 shutdown_connections 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f977803e510 0x7f97780409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 --2- 172.21.15.47:0/2967302616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f978c101e80 0x7f978c0ff300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 >> 172.21.15.47:0/2967302616 conn(0x7f978c0fd490 msgr2=0x7f978c06b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 shutdown_connections 2024-05-11T12:13:56.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:56.716+0000 7f979234c700 1 -- 172.21.15.47:0/2967302616 wait complete. 2024-05-11T12:13:57.495 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi050 2024-05-11T12:13:57.495 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:13:57.496 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.conf 2024-05-11T12:13:57.515 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:13:57.515 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:13:57.574 INFO:tasks.cephadm:Adding host smithi050 to orchestrator... 2024-05-11T12:13:57.574 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch host add smithi050 2024-05-11T12:13:57.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:57 smithi047 ceph-mon[30375]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:13:57.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:57.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:13:57.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:13:57.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:13:57.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:57 smithi047 ceph-mon[30375]: Updating smithi047:/etc/ceph/ceph.conf 2024-05-11T12:13:57.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:57 smithi047 ceph-mon[30375]: Updating smithi047:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:13:57.849 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:13:59.126 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:58 smithi047 ceph-mon[30375]: Updating smithi047:/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:13:59.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:58 smithi047 ceph-mon[30375]: Updating smithi047:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T12:13:59.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:59.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:59.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:13:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:13:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 -- 172.21.15.47:0/2686841396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 msgr2=0x7f6d381035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:13:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 --2- 172.21.15.47:0/2686841396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d381035a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009910 tx=0x7f6d28009c20 comp rx=0 tx=0).stop 2024-05-11T12:13:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 -- 172.21.15.47:0/2686841396 shutdown_connections 2024-05-11T12:13:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 --2- 172.21.15.47:0/2686841396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d381035a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:13:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 -- 172.21.15.47:0/2686841396 >> 172.21.15.47:0/2686841396 conn(0x7f6d380fea90 msgr2=0x7f6d38100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:13:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 -- 172.21.15.47:0/2686841396 shutdown_connections 2024-05-11T12:13:59.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 -- 172.21.15.47:0/2686841396 wait complete. 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.431+0000 7f6d3e56b700 1 Processor -- start 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d3e56b700 1 -- start start 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d3e56b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d38075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d3e56b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3810d440 con 0x7f6d381031c0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d38075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d37fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d38075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47040/0 (socket says 172.21.15.47:47040) 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d37fff700 1 -- 172.21.15.47:0/858723568 learned_addr learned my addr 172.21.15.47:0/858723568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d37fff700 1 -- 172.21.15.47:0/858723568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d38077740 con 0x7f6d381031c0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d37fff700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d38075850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6d280056d0 tx=0x7f6d280057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d35ffb700 1 -- 172.21.15.47:0/858723568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d28026070 con 0x7f6d381031c0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d35ffb700 1 -- 172.21.15.47:0/858723568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d2800bc00 con 0x7f6d381031c0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d38075df0 con 0x7f6d381031c0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.432+0000 7f6d35ffb700 1 -- 172.21.15.47:0/858723568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d28003920 con 0x7f6d381031c0 2024-05-11T12:13:59.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.433+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d380762b0 con 0x7f6d381031c0 2024-05-11T12:13:59.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.433+0000 7f6d35ffb700 1 -- 172.21.15.47:0/858723568 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6d28029070 con 0x7f6d381031c0 2024-05-11T12:13:59.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.433+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d381046a0 con 0x7f6d381031c0 2024-05-11T12:13:59.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.434+0000 7f6d35ffb700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6d2403e510 0x7f6d240409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:13:59.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.434+0000 7f6d35ffb700 1 -- 172.21.15.47:0/858723568 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d2801e070 con 0x7f6d381031c0 2024-05-11T12:13:59.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.434+0000 7f6d2ffff700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6d2403e510 0x7f6d240409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:13:59.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.434+0000 7f6d2ffff700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6d2403e510 0x7f6d240409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006fd0 tx=0x7f6d20006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:13:59.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.439+0000 7f6d35ffb700 1 -- 172.21.15.47:0/858723568 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6d28013070 con 0x7f6d381031c0 2024-05-11T12:13:59.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:13:59.659+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}) -- 0x7f6d38076540 con 0x7f6d2403e510 2024-05-11T12:14:00.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:00 smithi047 ceph-mon[30375]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:14:01.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:01 smithi047 ceph-mon[30375]: Deploying cephadm binary to smithi050 2024-05-11T12:14:02.524 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:02 smithi047 ceph-mon[30375]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:03.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.955+0000 7f6d35ffb700 1 -- 172.21.15.47:0/858723568 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6d38076540 con 0x7f6d2403e510 2024-05-11T12:14:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi050' with addr '172.21.15.50' 2024-05-11T12:14:03.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6d2403e510 msgr2=0x7f6d240409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6d2403e510 0x7f6d240409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006fd0 tx=0x7f6d20006e40 comp rx=0 tx=0).stop 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 msgr2=0x7f6d38075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d38075850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6d280056d0 tx=0x7f6d280057b0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 shutdown_connections 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6d2403e510 0x7f6d240409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 --2- 172.21.15.47:0/858723568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d381031c0 0x7f6d38075850 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 >> 172.21.15.47:0/858723568 conn(0x7f6d380fea90 msgr2=0x7f6d38104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 shutdown_connections 2024-05-11T12:14:03.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:03.958+0000 7f6d3e56b700 1 -- 172.21.15.47:0/858723568 wait complete. 2024-05-11T12:14:04.558 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-11T12:14:04.809 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:14:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:04 smithi047 ceph-mon[30375]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:04 smithi047 ceph-mon[30375]: Added host smithi050 2024-05-11T12:14:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:06.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:06.227 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:06 smithi047 ceph-mon[30375]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.465+0000 7fa40bf2d700 1 -- 172.21.15.47:0/15215512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 msgr2=0x7fa4041035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.465+0000 7fa40bf2d700 1 --2- 172.21.15.47:0/15215512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa4041035d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa400009910 tx=0x7fa400009c20 comp rx=0 tx=0).stop 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.465+0000 7fa40bf2d700 1 -- 172.21.15.47:0/15215512 shutdown_connections 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.465+0000 7fa40bf2d700 1 --2- 172.21.15.47:0/15215512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa4041035d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.465+0000 7fa40bf2d700 1 -- 172.21.15.47:0/15215512 >> 172.21.15.47:0/15215512 conn(0x7fa4040fea80 msgr2=0x7fa404100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.465+0000 7fa40bf2d700 1 -- 172.21.15.47:0/15215512 shutdown_connections 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.465+0000 7fa40bf2d700 1 -- 172.21.15.47:0/15215512 wait complete. 2024-05-11T12:14:06.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.466+0000 7fa40bf2d700 1 Processor -- start 2024-05-11T12:14:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.466+0000 7fa40bf2d700 1 -- start start 2024-05-11T12:14:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.466+0000 7fa40bf2d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa40419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.466+0000 7fa40bf2d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa40410d470 con 0x7fa4041031f0 2024-05-11T12:14:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.466+0000 7fa409cc9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa40419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.466+0000 7fa409cc9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa40419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47070/0 (socket says 172.21.15.47:47070) 2024-05-11T12:14:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.466+0000 7fa409cc9700 1 -- 172.21.15.47:0/576916948 learned_addr learned my addr 172.21.15.47:0/576916948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:14:06.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.467+0000 7fa409cc9700 1 -- 172.21.15.47:0/576916948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40419eb90 con 0x7fa4041031f0 2024-05-11T12:14:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.467+0000 7fa409cc9700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa40419e650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa400006010 tx=0x7fa400005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.467+0000 7fa3faffd700 1 -- 172.21.15.47:0/576916948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa400026070 con 0x7fa4041031f0 2024-05-11T12:14:06.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.467+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40419ee80 con 0x7fa4041031f0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.467+0000 7fa3faffd700 1 -- 172.21.15.47:0/576916948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40000bba0 con 0x7fa4041031f0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.467+0000 7fa3faffd700 1 -- 172.21.15.47:0/576916948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa400003910 con 0x7fa4041031f0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.467+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4041987d0 con 0x7fa4041031f0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.468+0000 7fa3faffd700 1 -- 172.21.15.47:0/576916948 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fa400029070 con 0x7fa4041031f0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.468+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4041046d0 con 0x7fa4041031f0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.468+0000 7fa3faffd700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa3f003e510 0x7fa3f00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.468+0000 7fa3faffd700 1 -- 172.21.15.47:0/576916948 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa40001e070 con 0x7fa4041031f0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.468+0000 7fa4094c8700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa3f003e510 0x7fa3f00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:06.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.469+0000 7fa4094c8700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa3f003e510 0x7fa3f00409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4006fd0 tx=0x7fa3f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:06.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.473+0000 7fa3faffd700 1 -- 172.21.15.47:0/576916948 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa400013070 con 0x7fa4041031f0 2024-05-11T12:14:06.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.692+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa404199d00 con 0x7fa3f003e510 2024-05-11T12:14:06.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.693+0000 7fa3faffd700 1 -- 172.21.15.47:0/576916948 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fa404199d00 con 0x7fa3f003e510 2024-05-11T12:14:06.694 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:14:06.694 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}] 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa3f003e510 msgr2=0x7fa3f00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa3f003e510 0x7fa3f00409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4006fd0 tx=0x7fa3f4006e40 comp rx=0 tx=0).stop 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 msgr2=0x7fa40419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa40419e650 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa400006010 tx=0x7fa400005780 comp rx=0 tx=0).stop 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 shutdown_connections 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa3f003e510 0x7fa3f00409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 --2- 172.21.15.47:0/576916948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa4041031f0 0x7fa40419e650 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 >> 172.21.15.47:0/576916948 conn(0x7fa4040fea80 msgr2=0x7fa404104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 shutdown_connections 2024-05-11T12:14:06.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:06.695+0000 7fa40bf2d700 1 -- 172.21.15.47:0/576916948 wait complete. 2024-05-11T12:14:07.434 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi064 2024-05-11T12:14:07.434 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:14:07.434 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.conf 2024-05-11T12:14:07.454 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:14:07.454 DEBUG:teuthology.orchestra.run.smithi064:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:14:07.512 INFO:tasks.cephadm:Adding host smithi064 to orchestrator... 2024-05-11T12:14:07.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch host add smithi064 2024-05-11T12:14:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:14:07.876 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:07 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:07.877 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:07 smithi047 ceph-mon[30375]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:14:08.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:08 smithi047 ceph-mon[30375]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.406+0000 7f7c4b75f700 1 -- 172.21.15.47:0/3907596594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 msgr2=0x7f7c44075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.406+0000 7f7c4b75f700 1 --2- 172.21.15.47:0/3907596594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c44075cd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000b0a0 tx=0x7f7c4000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.406+0000 7f7c4b75f700 1 -- 172.21.15.47:0/3907596594 shutdown_connections 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.406+0000 7f7c4b75f700 1 --2- 172.21.15.47:0/3907596594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c44075cd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.406+0000 7f7c4b75f700 1 -- 172.21.15.47:0/3907596594 >> 172.21.15.47:0/3907596594 conn(0x7f7c440fea90 msgr2=0x7f7c44100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.406+0000 7f7c4b75f700 1 -- 172.21.15.47:0/3907596594 shutdown_connections 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.406+0000 7f7c4b75f700 1 -- 172.21.15.47:0/3907596594 wait complete. 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.407+0000 7f7c4b75f700 1 Processor -- start 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.407+0000 7f7c4b75f700 1 -- start start 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.407+0000 7f7c4b75f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c4419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.407+0000 7f7c4b75f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44113ab0 con 0x7f7c440758f0 2024-05-11T12:14:09.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.407+0000 7f7c494fb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c4419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.407+0000 7f7c494fb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c4419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44976/0 (socket says 172.21.15.47:44976) 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.407+0000 7f7c494fb700 1 -- 172.21.15.47:0/323772850 learned_addr learned my addr 172.21.15.47:0/323772850 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.408+0000 7f7c494fb700 1 -- 172.21.15.47:0/323772850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4419f4b0 con 0x7f7c440758f0 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.408+0000 7f7c494fb700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c4419ef70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000d010 tx=0x7f7c40009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.408+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/323772850 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40021020 con 0x7f7c440758f0 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.408+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/323772850 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c40004000 con 0x7f7c440758f0 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.408+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4419f740 con 0x7f7c440758f0 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.408+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/323772850 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c40017040 con 0x7f7c440758f0 2024-05-11T12:14:09.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.408+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441a3840 con 0x7f7c440758f0 2024-05-11T12:14:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.409+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/323772850 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7c40024020 con 0x7f7c440758f0 2024-05-11T12:14:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.409+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c44077030 con 0x7f7c440758f0 2024-05-11T12:14:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.409+0000 7f7c3a7fc700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c3003e560 0x7f7c30040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.410+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/323772850 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7c4000f050 con 0x7f7c440758f0 2024-05-11T12:14:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.410+0000 7f7c48cfa700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c3003e560 0x7f7c30040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:09.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.410+0000 7f7c48cfa700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c3003e560 0x7f7c30040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006fd0 tx=0x7f7c34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:09.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.414+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/323772850 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c4002b020 con 0x7f7c440758f0 2024-05-11T12:14:09.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:09.631+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}) -- 0x7f7c44064bb0 con 0x7f7c3003e560 2024-05-11T12:14:11.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi064", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:10 smithi047 ceph-mon[30375]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-11T12:14:12.175 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:11 smithi047 ceph-mon[30375]: Updating smithi050:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:12.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:11 smithi047 ceph-mon[30375]: Deploying cephadm binary to smithi064 2024-05-11T12:14:12.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:11 smithi047 ceph-mon[30375]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:14:12.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:11 smithi047 ceph-mon[30375]: Updating smithi050:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T12:14:13.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:12 smithi047 ceph-mon[30375]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:13.871 INFO:teuthology.orchestra.run.smithi047.stdout:Added host 'smithi064' with addr '172.21.15.64' 2024-05-11T12:14:13.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.870+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/323772850 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f7c44064bb0 con 0x7f7c3003e560 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c3003e560 msgr2=0x7f7c30040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c3003e560 0x7f7c30040a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006fd0 tx=0x7f7c34006e40 comp rx=0 tx=0).stop 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 msgr2=0x7f7c4419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c4419ef70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000d010 tx=0x7f7c40009790 comp rx=0 tx=0).stop 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 shutdown_connections 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c3003e560 0x7f7c30040a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 --2- 172.21.15.47:0/323772850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c440758f0 0x7f7c4419ef70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 >> 172.21.15.47:0/323772850 conn(0x7f7c440fea90 msgr2=0x7f7c440fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 shutdown_connections 2024-05-11T12:14:13.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:13.873+0000 7f7c4b75f700 1 -- 172.21.15.47:0/323772850 wait complete. 2024-05-11T12:14:14.448 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:14.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:14.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:14.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:14 smithi047 ceph-mon[30375]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:14.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:14.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:14 smithi047 ceph-mon[30375]: Added host smithi064 2024-05-11T12:14:14.449 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:14.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch host ls --format=json 2024-05-11T12:14:14.703 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:14:15.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:15 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:15.610 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:15 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:16.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.327+0000 7f5666dcc700 1 -- 172.21.15.47:0/2088576675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 msgr2=0x7f56601024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.327+0000 7f5666dcc700 1 --2- 172.21.15.47:0/2088576675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f56601024a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5650009910 tx=0x7f5650009c20 comp rx=0 tx=0).stop 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.327+0000 7f5666dcc700 1 -- 172.21.15.47:0/2088576675 shutdown_connections 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.327+0000 7f5666dcc700 1 --2- 172.21.15.47:0/2088576675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f56601024a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.327+0000 7f5666dcc700 1 -- 172.21.15.47:0/2088576675 >> 172.21.15.47:0/2088576675 conn(0x7f56600fd950 msgr2=0x7f56600ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.327+0000 7f5666dcc700 1 -- 172.21.15.47:0/2088576675 shutdown_connections 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.327+0000 7f5666dcc700 1 -- 172.21.15.47:0/2088576675 wait complete. 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.328+0000 7f5666dcc700 1 Processor -- start 2024-05-11T12:14:16.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.328+0000 7f5666dcc700 1 -- start start 2024-05-11T12:14:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.328+0000 7f5666dcc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f566019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.328+0000 7f5666dcc700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f566010c340 con 0x7f56601020c0 2024-05-11T12:14:16.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.328+0000 7f5664b68700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f566019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.328+0000 7f5664b68700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f566019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44994/0 (socket says 172.21.15.47:44994) 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.328+0000 7f5664b68700 1 -- 172.21.15.47:0/3575436334 learned_addr learned my addr 172.21.15.47:0/3575436334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.329+0000 7f5664b68700 1 -- 172.21.15.47:0/3575436334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566019eb90 con 0x7f56601020c0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.329+0000 7f5664b68700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f566019e650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5650006010 tx=0x7f5650005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.329+0000 7f565dffb700 1 -- 172.21.15.47:0/3575436334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5650026070 con 0x7f56601020c0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.329+0000 7f565dffb700 1 -- 172.21.15.47:0/3575436334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565000bba0 con 0x7f56601020c0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.329+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566019ee80 con 0x7f56601020c0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.329+0000 7f565dffb700 1 -- 172.21.15.47:0/3575436334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5650003910 con 0x7f56601020c0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.329+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56601987d0 con 0x7f56601020c0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.330+0000 7f565dffb700 1 -- 172.21.15.47:0/3575436334 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5650029070 con 0x7f56601020c0 2024-05-11T12:14:16.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.330+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56601035a0 con 0x7f56601020c0 2024-05-11T12:14:16.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.330+0000 7f565dffb700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f564803e510 0x7f56480409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:16.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.330+0000 7f565dffb700 1 -- 172.21.15.47:0/3575436334 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f565001e070 con 0x7f56601020c0 2024-05-11T12:14:16.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.331+0000 7f565ffff700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f564803e510 0x7f56480409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:16.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.331+0000 7f565ffff700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f564803e510 0x7f56480409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5654006fd0 tx=0x7f5654006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:16.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.336+0000 7f565dffb700 1 -- 172.21.15.47:0/3575436334 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5650013070 con 0x7f56601020c0 2024-05-11T12:14:16.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.560+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5660199d00 con 0x7f564803e510 2024-05-11T12:14:16.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.561+0000 7f565dffb700 1 -- 172.21.15.47:0/3575436334 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f5660199d00 con 0x7f564803e510 2024-05-11T12:14:16.561 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:14:16.562 INFO:teuthology.orchestra.run.smithi047.stdout:[{"addr": "172.21.15.47", "hostname": "smithi047", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}, {"addr": "172.21.15.64", "hostname": "smithi064", "labels": [], "status": ""}] 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f564803e510 msgr2=0x7f56480409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f564803e510 0x7f56480409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5654006fd0 tx=0x7f5654006e40 comp rx=0 tx=0).stop 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 msgr2=0x7f566019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f566019e650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5650006010 tx=0x7f5650005780 comp rx=0 tx=0).stop 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 shutdown_connections 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f564803e510 0x7f56480409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 --2- 172.21.15.47:0/3575436334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f56601020c0 0x7f566019e650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 >> 172.21.15.47:0/3575436334 conn(0x7f56600fd950 msgr2=0x7f566006b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:16.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 shutdown_connections 2024-05-11T12:14:16.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:16.563+0000 7f5666dcc700 1 -- 172.21.15.47:0/3575436334 wait complete. 2024-05-11T12:14:16.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:16 smithi047 ceph-mon[30375]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:17.265 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-11T12:14:17.266 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd crush tunables default 2024-05-11T12:14:17.517 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:14:17.902 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:17 smithi047 ceph-mon[30375]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:14:17.902 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:18.923 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:18 smithi047 ceph-mon[30375]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.153+0000 7f516b235700 1 -- 172.21.15.47:0/1831940363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 msgr2=0x7f51641035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.153+0000 7f516b235700 1 --2- 172.21.15.47:0/1831940363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f51641035a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5160009910 tx=0x7f5160009c20 comp rx=0 tx=0).stop 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.153+0000 7f516b235700 1 -- 172.21.15.47:0/1831940363 shutdown_connections 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.153+0000 7f516b235700 1 --2- 172.21.15.47:0/1831940363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f51641035a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.153+0000 7f516b235700 1 -- 172.21.15.47:0/1831940363 >> 172.21.15.47:0/1831940363 conn(0x7f51640fea90 msgr2=0x7f5164100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.153+0000 7f516b235700 1 -- 172.21.15.47:0/1831940363 shutdown_connections 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.153+0000 7f516b235700 1 -- 172.21.15.47:0/1831940363 wait complete. 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f516b235700 1 Processor -- start 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f516b235700 1 -- start start 2024-05-11T12:14:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f516b235700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f516419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f516b235700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f516410d440 con 0x7f51641031c0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f5168fd1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f516419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f5168fd1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f516419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37158/0 (socket says 172.21.15.47:37158) 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f5168fd1700 1 -- 172.21.15.47:0/1488354763 learned_addr learned my addr 172.21.15.47:0/1488354763 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f5168fd1700 1 -- 172.21.15.47:0/1488354763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516419eb70 con 0x7f51641031c0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.154+0000 7f5168fd1700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f516419e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5160006010 tx=0x7f5160005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.155+0000 7f5159ffb700 1 -- 172.21.15.47:0/1488354763 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5160026070 con 0x7f51641031c0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.155+0000 7f5159ffb700 1 -- 172.21.15.47:0/1488354763 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f516000bba0 con 0x7f51641031c0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.155+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516419ee60 con 0x7f51641031c0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.155+0000 7f5159ffb700 1 -- 172.21.15.47:0/1488354763 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5160003910 con 0x7f51641031c0 2024-05-11T12:14:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.155+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641987b0 con 0x7f51641031c0 2024-05-11T12:14:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.156+0000 7f5159ffb700 1 -- 172.21.15.47:0/1488354763 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5160029070 con 0x7f51641031c0 2024-05-11T12:14:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.156+0000 7f5159ffb700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f514c03e570 0x7f514c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.156+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51641046a0 con 0x7f51641031c0 2024-05-11T12:14:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.156+0000 7f5159ffb700 1 -- 172.21.15.47:0/1488354763 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f516001e070 con 0x7f51641031c0 2024-05-11T12:14:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.156+0000 7f515bfff700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f514c03e570 0x7f514c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.157+0000 7f515bfff700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f514c03e570 0x7f514c040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5154006fd0 tx=0x7f5154006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:19.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.161+0000 7f5159ffb700 1 -- 172.21.15.47:0/1488354763 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5160030020 con 0x7f51641031c0 2024-05-11T12:14:19.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.379+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5164199ce0 con 0x7f51641031c0 2024-05-11T12:14:19.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.593+0000 7f5159ffb700 1 -- 172.21.15.47:0/1488354763 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5160013070 con 0x7f51641031c0 2024-05-11T12:14:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f514c03e570 msgr2=0x7f514c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f514c03e570 0x7f514c040a30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5154006fd0 tx=0x7f5154006e40 comp rx=0 tx=0).stop 2024-05-11T12:14:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 msgr2=0x7f516419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f516419e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5160006010 tx=0x7f5160005780 comp rx=0 tx=0).stop 2024-05-11T12:14:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 shutdown_connections 2024-05-11T12:14:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f514c03e570 0x7f514c040a30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 --2- 172.21.15.47:0/1488354763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51641031c0 0x7f516419e630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 >> 172.21.15.47:0/1488354763 conn(0x7f51640fea90 msgr2=0x7f5164104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 shutdown_connections 2024-05-11T12:14:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:19.596+0000 7f516b235700 1 -- 172.21.15.47:0/1488354763 wait complete. 2024-05-11T12:14:19.598 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-05-11T12:14:19.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:19 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1488354763' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-11T12:14:20.406 INFO:tasks.cephadm:Adding mon.a on smithi047 2024-05-11T12:14:20.406 INFO:tasks.cephadm:Adding mon.b on smithi050 2024-05-11T12:14:20.406 INFO:tasks.cephadm:Adding mon.c on smithi064 2024-05-11T12:14:20.407 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch apply mon '3;smithi047:172.21.15.47=a;smithi050:172.21.15.50=b;smithi064:172.21.15.64=c' 2024-05-11T12:14:20.644 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1488354763' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi064", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:20.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:20.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-11T12:14:20.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:20 smithi047 ceph-mon[30375]: Updating smithi064:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:22.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:21 smithi047 ceph-mon[30375]: Updating smithi064:/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:14:22.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:21 smithi047 ceph-mon[30375]: Updating smithi064:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.client.admin.keyring 2024-05-11T12:14:22.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:22.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:22.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:22.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.053+0000 7f0bf6c61700 1 -- 172.21.15.64:0/3607088430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 msgr2=0x7f0bf01035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.053+0000 7f0bf6c61700 1 --2- 172.21.15.64:0/3607088430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf01035a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0be4009910 tx=0x7f0be4009c20 comp rx=0 tx=0).stop 2024-05-11T12:14:22.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.053+0000 7f0bf6c61700 1 -- 172.21.15.64:0/3607088430 shutdown_connections 2024-05-11T12:14:22.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.053+0000 7f0bf6c61700 1 --2- 172.21.15.64:0/3607088430 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf01035a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.131 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.053+0000 7f0bf6c61700 1 -- 172.21.15.64:0/3607088430 >> 172.21.15.64:0/3607088430 conn(0x7f0bf00fea90 msgr2=0x7f0bf0100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.053+0000 7f0bf6c61700 1 -- 172.21.15.64:0/3607088430 shutdown_connections 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.053+0000 7f0bf6c61700 1 -- 172.21.15.64:0/3607088430 wait complete. 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf6c61700 1 Processor -- start 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf6c61700 1 -- start start 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf6c61700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf6c61700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf010d440 con 0x7f0bf01031c0 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf49fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf49fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf019e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.64:58260/0 (socket says 172.21.15.64:58260) 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf49fd700 1 -- 172.21.15.64:0/52544659 learned_addr learned my addr 172.21.15.64:0/52544659 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.054+0000 7f0bf49fd700 1 -- 172.21.15.64:0/52544659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf019eb70 con 0x7f0bf01031c0 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.055+0000 7f0bf49fd700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf019e630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be40056d0 tx=0x7f0be40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.055+0000 7f0bedffb700 1 -- 172.21.15.64:0/52544659 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0be4026070 con 0x7f0bf01031c0 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.055+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf019ee60 con 0x7f0bf01031c0 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.055+0000 7f0bedffb700 1 -- 172.21.15.64:0/52544659 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0be400bba0 con 0x7f0bf01031c0 2024-05-11T12:14:22.132 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.055+0000 7f0bedffb700 1 -- 172.21.15.64:0/52544659 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0be4003920 con 0x7f0bf01031c0 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.055+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf01987b0 con 0x7f0bf01031c0 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.056+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf01046a0 con 0x7f0bf01031c0 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.056+0000 7f0bedffb700 1 -- 172.21.15.64:0/52544659 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0be4029070 con 0x7f0bf01031c0 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.056+0000 7f0bedffb700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0be003e510 0x7f0be00409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.056+0000 7f0bedffb700 1 -- 172.21.15.64:0/52544659 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0be401e070 con 0x7f0bf01031c0 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.056+0000 7f0beffff700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0be003e510 0x7f0be00409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.057+0000 7f0beffff700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0be003e510 0x7f0be00409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006fd0 tx=0x7f0bdc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:22.133 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.062+0000 7f0bedffb700 1 -- 172.21.15.64:0/52544659 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0be4013070 con 0x7f0bf01031c0 2024-05-11T12:14:22.290 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.291+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi050:172.21.15.50=b;smithi064:172.21.15.64=c", "target": ["mon-mgr", ""]}) -- 0x7f0bf0199d90 con 0x7f0be003e510 2024-05-11T12:14:22.297 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.298+0000 7f0bedffb700 1 -- 172.21.15.64:0/52544659 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0bf0199d90 con 0x7f0be003e510 2024-05-11T12:14:22.297 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mon update... 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0be003e510 msgr2=0x7f0be00409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0be003e510 0x7f0be00409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc006fd0 tx=0x7f0bdc006e40 comp rx=0 tx=0).stop 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 msgr2=0x7f0bf019e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf019e630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0be40056d0 tx=0x7f0be40057b0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 shutdown_connections 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0be003e510 0x7f0be00409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 --2- 172.21.15.64:0/52544659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0bf01031c0 0x7f0bf019e630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 >> 172.21.15.64:0/52544659 conn(0x7f0bf00fea90 msgr2=0x7f0bf0104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 shutdown_connections 2024-05-11T12:14:22.300 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:22.300+0000 7f0bf6c61700 1 -- 172.21.15.64:0/52544659 wait complete. 2024-05-11T12:14:22.808 DEBUG:teuthology.orchestra.run.smithi050:mon.b> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.b.service 2024-05-11T12:14:22.811 DEBUG:teuthology.orchestra.run.smithi064:mon.c> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.c.service 2024-05-11T12:14:22.813 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T12:14:22.813 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T12:14:22.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:22.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:22.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:22.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:22.922 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:22.923 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:22.923 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:22.923 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:23.443 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:23.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:23 smithi047 ceph-mon[30375]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi047:172.21.15.47=a;smithi050:172.21.15.50=b;smithi064:172.21.15.64=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:14:23.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:23 smithi047 ceph-mon[30375]: Saving service mon spec with placement smithi047:172.21.15.47=a;smithi050:172.21.15.50=b;smithi064:172.21.15.64=c;count:3 2024-05-11T12:14:23.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:23 smithi047 ceph-mon[30375]: Deploying daemon mon.c on smithi064 2024-05-11T12:14:25.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:24 smithi047 ceph-mon[30375]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:26.375 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.374+0000 7f6947cd5700 1 -- 172.21.15.64:0/2223026826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 msgr2=0x7f694006afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.374+0000 7f6947cd5700 1 --2- 172.21.15.64:0/2223026826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694006afb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6930009910 tx=0x7f6930009c20 comp rx=0 tx=0).stop 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.374+0000 7f6947cd5700 1 -- 172.21.15.64:0/2223026826 shutdown_connections 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.374+0000 7f6947cd5700 1 --2- 172.21.15.64:0/2223026826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694006afb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.374+0000 7f6947cd5700 1 -- 172.21.15.64:0/2223026826 >> 172.21.15.64:0/2223026826 conn(0x7f69400779e0 msgr2=0x7f6940077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.375+0000 7f6947cd5700 1 -- 172.21.15.64:0/2223026826 shutdown_connections 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.375+0000 7f6947cd5700 1 -- 172.21.15.64:0/2223026826 wait complete. 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.375+0000 7f6947cd5700 1 Processor -- start 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.375+0000 7f6947cd5700 1 -- start start 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.375+0000 7f6947cd5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.375+0000 7f6947cd5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f694010d440 con 0x7f694006abd0 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6945a71700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6945a71700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694019e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.64:58280/0 (socket says 172.21.15.64:58280) 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6945a71700 1 -- 172.21.15.64:0/973863697 learned_addr learned my addr 172.21.15.64:0/973863697 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:14:26.376 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6945a71700 1 -- 172.21.15.64:0/973863697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694019ebe0 con 0x7f694006abd0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6945a71700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694019e6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69300056d0 tx=0x7f69300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6936ffd700 1 -- 172.21.15.64:0/973863697 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6930026070 con 0x7f694006abd0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694019eed0 con 0x7f694006abd0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6936ffd700 1 -- 172.21.15.64:0/973863697 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693000bba0 con 0x7f694006abd0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.377+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6940198860 con 0x7f694006abd0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.376+0000 7f6936ffd700 1 -- 172.21.15.64:0/973863697 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6930003920 con 0x7f694006abd0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.377+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69401046d0 con 0x7f694006abd0 2024-05-11T12:14:26.378 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.377+0000 7f6936ffd700 1 -- 172.21.15.64:0/973863697 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f6930029070 con 0x7f694006abd0 2024-05-11T12:14:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.378+0000 7f6936ffd700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f692c03e510 0x7f692c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.378+0000 7f6936ffd700 1 -- 172.21.15.64:0/973863697 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f693001e070 con 0x7f694006abd0 2024-05-11T12:14:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.378+0000 7f6945270700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f692c03e510 0x7f692c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:26.379 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.378+0000 7f6945270700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f692c03e510 0x7f692c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f693c006fd0 tx=0x7f693c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:26.383 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.383+0000 7f6936ffd700 1 -- 172.21.15.64:0/973863697 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6930013070 con 0x7f694006abd0 2024-05-11T12:14:26.707 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.707+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6940199d90 con 0x7f694006abd0 2024-05-11T12:14:26.708 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.708+0000 7f6936ffd700 1 -- 172.21.15.64:0/973863697 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f6930030090 con 0x7f694006abd0 2024-05-11T12:14:26.708 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:14:26.708 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":1,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","modified":"2024-05-11T12:12:33.150058Z","created":"2024-05-11T12:12:33.150058Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-11T12:14:26.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.716+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f692c03e510 msgr2=0x7f692c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.716+0000 7f6947cd5700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f692c03e510 0x7f692c0409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f693c006fd0 tx=0x7f693c006e40 comp rx=0 tx=0).stop 2024-05-11T12:14:26.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.716+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 msgr2=0x7f694019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:26.716 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.716+0000 7f6947cd5700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694019e6a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f69300056d0 tx=0x7f69300057b0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.719+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 shutdown_connections 2024-05-11T12:14:26.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.719+0000 7f6947cd5700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f692c03e510 0x7f692c0409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.719+0000 7f6947cd5700 1 --2- 172.21.15.64:0/973863697 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f694006abd0 0x7f694019e6a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:26.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.719+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 >> 172.21.15.64:0/973863697 conn(0x7f69400779e0 msgr2=0x7f6940104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:26.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.720+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 shutdown_connections 2024-05-11T12:14:26.719 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:26.720+0000 7f6947cd5700 1 -- 172.21.15.64:0/973863697 wait complete. 2024-05-11T12:14:26.720 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 1 2024-05-11T12:14:27.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:26 smithi047 ceph-mon[30375]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:27 smithi064 systemd[1]: Started Ceph mon.c for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:14:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:27 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/973863697' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T12:14:28.322 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T12:14:28.322 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T12:14:28.575 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:14:33.045 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.044+0000 7f0d90d21700 1 -- 172.21.15.64:0/1298313109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 msgr2=0x7f0d74005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.044+0000 7f0d90d21700 1 --2- 172.21.15.64:0/1298313109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 0x7f0d74005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c06b300 tx=0x7f0d7c00fe60 comp rx=0 tx=0).stop 2024-05-11T12:14:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.044+0000 7f0d90d21700 1 -- 172.21.15.64:0/1298313109 shutdown_connections 2024-05-11T12:14:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.044+0000 7f0d90d21700 1 --2- 172.21.15.64:0/1298313109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 0x7f0d74005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.044+0000 7f0d90d21700 1 -- 172.21.15.64:0/1298313109 >> 172.21.15.64:0/1298313109 conn(0x7f0d8c078370 msgr2=0x7f0d8c078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.044+0000 7f0d90d21700 1 -- 172.21.15.64:0/1298313109 shutdown_connections 2024-05-11T12:14:33.046 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.044+0000 7f0d90d21700 1 -- 172.21.15.64:0/1298313109 wait complete. 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.045+0000 7f0d90d21700 1 Processor -- start 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.045+0000 7f0d90d21700 1 -- start start 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d90d21700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 0x7f0d8c1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d90d21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 0x7f0d8c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d90d21700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c10d1e0 con 0x7f0d740035c0 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d90d21700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d8c10d060 con 0x7f0d8c1a0e70 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d89d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 0x7f0d8c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d89d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 0x7f0d8c19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50836/0 (socket says 172.21.15.64:50836) 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d89d9b700 1 -- 172.21.15.64:0/3274417890 learned_addr learned my addr 172.21.15.64:0/3274417890 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d8a59c700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 0x7f0d8c1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:33.048 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d89d9b700 1 -- 172.21.15.64:0/3274417890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 msgr2=0x7f0d8c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d89d9b700 1 -- 172.21.15.64:0/3274417890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 msgr2=0x7f0d8c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d89d9b700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 0x7f0d8c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d89d9b700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 0x7f0d8c19aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d8a59c700 1 -- 172.21.15.64:0/3274417890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 msgr2=0x7f0d8c19aaf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d8a59c700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 0x7f0d8c19aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.046+0000 7f0d8a59c700 1 -- 172.21.15.64:0/3274417890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8c1a14b0 con 0x7f0d740035c0 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.047+0000 7f0d8a59c700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 0x7f0d8c1a0930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c06b300 tx=0x7f0d7c013c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.047+0000 7f0d7b7fe700 1 -- 172.21.15.64:0/3274417890 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0d7c003d00 con 0x7f0d740035c0 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.047+0000 7f0d7b7fe700 1 -- 172.21.15.64:0/3274417890 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d7c01c550 con 0x7f0d740035c0 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.047+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8c19b030 con 0x7f0d740035c0 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.047+0000 7f0d7b7fe700 1 -- 172.21.15.64:0/3274417890 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0d7c011670 con 0x7f0d740035c0 2024-05-11T12:14:33.049 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.047+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8c19b640 con 0x7f0d740035c0 2024-05-11T12:14:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.048+0000 7f0d7b7fe700 1 -- 172.21.15.64:0/3274417890 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0d7c004350 con 0x7f0d740035c0 2024-05-11T12:14:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.048+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d8c19b8f0 con 0x7f0d740035c0 2024-05-11T12:14:33.050 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.048+0000 7f0d7b7fe700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d6c03e320 0x7f0d6c0407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.049+0000 7f0d7b7fe700 1 -- 172.21.15.64:0/3274417890 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0d7c024070 con 0x7f0d740035c0 2024-05-11T12:14:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.049+0000 7f0d89d9b700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d6c03e320 0x7f0d6c0407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:33.051 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.049+0000 7f0d89d9b700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d6c03e320 0x7f0d6c0407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009180 tx=0x7f0d80008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:33.054 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.054+0000 7f0d7b7fe700 1 -- 172.21.15.64:0/3274417890 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d7c00ee10 con 0x7f0d740035c0 2024-05-11T12:14:33.091 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:33.091 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:14:33.091 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: mon.a calling monitor election 2024-05-11T12:14:33.091 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.091 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.091 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:33.091 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: mon.c calling monitor election 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: monmap epoch 2 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: last_changed 2024-05-11T12:14:27.765808+0000 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: created 2024-05-11T12:12:33.150058+0000 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: min_mon_release 19 (squid) 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: election_strategy: 1 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-11T12:14:33.092 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: fsmap 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: mgrmap e13: x(active, since 51s) 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: overall HEALTH_OK 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:33.093 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:33.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:33.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:14:33.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: mon.a calling monitor election 2024-05-11T12:14:33.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: mon.c calling monitor election 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: monmap epoch 2 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:14:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: last_changed 2024-05-11T12:14:27.765808+0000 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: created 2024-05-11T12:12:33.150058+0000 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: min_mon_release 19 (squid) 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: election_strategy: 1 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: fsmap 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: mgrmap e13: x(active, since 51s) 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: overall HEALTH_OK 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:33.363 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.362+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0d8c19bfe0 con 0x7f0d740035c0 2024-05-11T12:14:33.364 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.363+0000 7f0d7b7fe700 1 -- 172.21.15.64:0/3274417890 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f0d7c01c700 con 0x7f0d740035c0 2024-05-11T12:14:33.364 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:14:33.364 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","modified":"2024-05-11T12:14:27.765808Z","created":"2024-05-11T12:12:33.150058Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-11T12:14:33.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d6c03e320 msgr2=0x7f0d6c0407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d6c03e320 0x7f0d6c0407e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009180 tx=0x7f0d80008040 comp rx=0 tx=0).stop 2024-05-11T12:14:33.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 msgr2=0x7f0d8c1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:33.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 0x7f0d8c1a0930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c06b300 tx=0x7f0d7c013c40 comp rx=0 tx=0).stop 2024-05-11T12:14:33.367 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 shutdown_connections 2024-05-11T12:14:33.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d6c03e320 0x7f0d6c0407e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d740035c0 0x7f0d8c1a0930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 --2- 172.21.15.64:0/3274417890 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d8c1a0e70 0x7f0d8c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:33.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 >> 172.21.15.64:0/3274417890 conn(0x7f0d8c078370 msgr2=0x7f0d8c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:33.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 shutdown_connections 2024-05-11T12:14:33.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:33.366+0000 7f0d90d21700 1 -- 172.21.15.64:0/3274417890 wait complete. 2024-05-11T12:14:33.368 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 2 2024-05-11T12:14:33.925 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:33 smithi064 ceph-mon[34537]: Deploying daemon mon.b on smithi050 2024-05-11T12:14:33.925 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:33 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/3274417890' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T12:14:33.925 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:33 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:34.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:33 smithi047 ceph-mon[30375]: Deploying daemon mon.b on smithi050 2024-05-11T12:14:34.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:33 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/3274417890' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T12:14:34.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:33 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:34.953 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T12:14:34.954 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T12:14:35.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:34 smithi064 ceph-mon[34537]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:35.208 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:14:35.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:34 smithi047 ceph-mon[30375]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.469+0000 7fb01108b700 1 -- 172.21.15.64:0/1740518248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 msgr2=0x7fb00c1036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.469+0000 7fb01108b700 1 --2- 172.21.15.64:0/1740518248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 0x7fb00c1036b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb000009910 tx=0x7fb000009c20 comp rx=0 tx=0).stop 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.470+0000 7fb01108b700 1 -- 172.21.15.64:0/1740518248 shutdown_connections 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.470+0000 7fb01108b700 1 --2- 172.21.15.64:0/1740518248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 0x7fb00c1036b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.470+0000 7fb01108b700 1 -- 172.21.15.64:0/1740518248 >> 172.21.15.64:0/1740518248 conn(0x7fb00c0feb60 msgr2=0x7fb00c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.470+0000 7fb01108b700 1 -- 172.21.15.64:0/1740518248 shutdown_connections 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.470+0000 7fb01108b700 1 -- 172.21.15.64:0/1740518248 wait complete. 2024-05-11T12:14:36.474 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.470+0000 7fb01108b700 1 Processor -- start 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb01108b700 1 -- start start 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb01108b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 0x7fb00c075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb01108b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 0x7fb00c076210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb01108b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c10d550 con 0x7fb00c1032d0 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb01108b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c10d3d0 con 0x7fb00c075d90 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 0x7fb00c076210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 0x7fb00c076210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50846/0 (socket says 172.21.15.64:50846) 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 0x7fb00c075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00a59c700 1 -- 172.21.15.64:0/498849539 learned_addr learned my addr 172.21.15.64:0/498849539 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00a59c700 1 -- 172.21.15.64:0/498849539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 msgr2=0x7fb00c076210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00a59c700 1 -- 172.21.15.64:0/498849539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 msgr2=0x7fb00c076210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-11T12:14:36.475 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00a59c700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 0x7fb00c076210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00a59c700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 0x7fb00c076210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00ad9d700 1 -- 172.21.15.64:0/498849539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 msgr2=0x7fb00c076210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00ad9d700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 0x7fb00c076210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.471+0000 7fb00ad9d700 1 -- 172.21.15.64:0/498849539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c077770 con 0x7fb00c1032d0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.472+0000 7fb00ad9d700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 0x7fb00c075850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb000014000 tx=0x7fb000005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.472+0000 7faff3fff700 1 -- 172.21.15.64:0/498849539 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb000005dc0 con 0x7fb00c1032d0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.472+0000 7faff3fff700 1 -- 172.21.15.64:0/498849539 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb00000b6f0 con 0x7fb00c1032d0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.472+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1a4f10 con 0x7fb00c1032d0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.472+0000 7faff3fff700 1 -- 172.21.15.64:0/498849539 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb0000038f0 con 0x7fb00c1032d0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.472+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a53c0 con 0x7fb00c1032d0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.473+0000 7faff3fff700 1 -- 172.21.15.64:0/498849539 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb000029070 con 0x7fb00c1032d0 2024-05-11T12:14:36.476 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.473+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c1047b0 con 0x7fb00c1032d0 2024-05-11T12:14:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.473+0000 7faff3fff700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faff403e2b0 0x7faff4040770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.473+0000 7faff3fff700 1 -- 172.21.15.64:0/498849539 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb00001e070 con 0x7fb00c1032d0 2024-05-11T12:14:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.473+0000 7fb00a59c700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faff403e2b0 0x7faff4040770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:36.477 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.474+0000 7fb00a59c700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faff403e2b0 0x7faff4040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faffc009180 tx=0x7faffc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:36.479 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.479+0000 7faff3fff700 1 -- 172.21.15.64:0/498849539 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb000013070 con 0x7fb00c1032d0 2024-05-11T12:14:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.780+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb00c1a5710 con 0x7fb00c1032d0 2024-05-11T12:14:36.781 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.781+0000 7faff3fff700 1 -- 172.21.15.64:0/498849539 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fb000030090 con 0x7fb00c1032d0 2024-05-11T12:14:36.782 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:14:36.782 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":2,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","modified":"2024-05-11T12:14:27.765808Z","created":"2024-05-11T12:12:33.150058Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-11T12:14:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faff403e2b0 msgr2=0x7faff4040770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faff403e2b0 0x7faff4040770 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faffc009180 tx=0x7faffc008040 comp rx=0 tx=0).stop 2024-05-11T12:14:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 msgr2=0x7fb00c075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 0x7fb00c075850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb000014000 tx=0x7fb000005030 comp rx=0 tx=0).stop 2024-05-11T12:14:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 shutdown_connections 2024-05-11T12:14:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faff403e2b0 0x7faff4040770 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.784 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb00c1032d0 0x7fb00c075850 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 --2- 172.21.15.64:0/498849539 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb00c075d90 0x7fb00c076210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:36.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 >> 172.21.15.64:0/498849539 conn(0x7fb00c0feb60 msgr2=0x7fb00c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:36.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 shutdown_connections 2024-05-11T12:14:36.785 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:36.783+0000 7fb01108b700 1 -- 172.21.15.64:0/498849539 wait complete. 2024-05-11T12:14:36.785 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 2 2024-05-11T12:14:36.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:36 smithi050 systemd[1]: Started Ceph mon.b for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:14:38.307 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-11T12:14:38.307 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph mon dump -f json 2024-05-11T12:14:38.560 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: mon.a calling monitor election 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: mon.c calling monitor election 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: mon.b calling monitor election 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: monmap epoch 3 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: last_changed 2024-05-11T12:14:36.832606+0000 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: created 2024-05-11T12:12:33.150058+0000 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: min_mon_release 19 (squid) 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: election_strategy: 1 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-11T12:14:42.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: fsmap 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: mgrmap e13: x(active, since 60s) 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: overall HEALTH_OK 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: mon.a calling monitor election 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: mon.c calling monitor election 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: mon.b calling monitor election 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: monmap epoch 3 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: last_changed 2024-05-11T12:14:36.832606+0000 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: created 2024-05-11T12:12:33.150058+0000 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: min_mon_release 19 (squid) 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: election_strategy: 1 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-05-11T12:14:42.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: fsmap 2024-05-11T12:14:42.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T12:14:42.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: mgrmap e13: x(active, since 60s) 2024-05-11T12:14:42.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: overall HEALTH_OK 2024-05-11T12:14:42.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: mon.a calling monitor election 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: mon.c calling monitor election 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: mon.b calling monitor election 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: monmap epoch 3 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: last_changed 2024-05-11T12:14:36.832606+0000 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: created 2024-05-11T12:12:33.150058+0000 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: min_mon_release 19 (squid) 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: election_strategy: 1 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: 0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: 1: [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] mon.c 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: 2: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: fsmap 2024-05-11T12:14:42.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: osdmap e4: 0 total, 0 up, 0 in 2024-05-11T12:14:42.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: mgrmap e13: x(active, since 60s) 2024-05-11T12:14:42.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: overall HEALTH_OK 2024-05-11T12:14:42.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.366 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.991+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3467849655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef01021a0 msgr2=0x7f5edc005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.991+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3467849655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef01021a0 0x7f5edc005610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8005fa0 tx=0x7f5ed800ffc0 comp rx=0 tx=0).stop 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.992+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3467849655 shutdown_connections 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.992+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3467849655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef01021a0 0x7f5edc005610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.992+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3467849655 >> 172.21.15.64:0/3467849655 conn(0x7f5ef00fda30 msgr2=0x7f5ef00ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.992+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3467849655 shutdown_connections 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.992+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3467849655 wait complete. 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.992+0000 7f5ef68e2700 1 Processor -- start 2024-05-11T12:14:42.993 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.992+0000 7f5ef68e2700 1 -- start start 2024-05-11T12:14:42.995 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef68e2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ef01021a0 0x7f5ef019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef68e2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef019ed20 0x7f5ef01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef68e2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef0198f30 0x7f5ef01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef68e2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0109e30 con 0x7f5ef019ed20 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef68e2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0109cb0 con 0x7f5ef01021a0 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef68e2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0109fb0 con 0x7f5ef0198f30 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef4e7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef0198f30 0x7f5ef01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eef7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef019ed20 0x7f5ef01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef4e7f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef0198f30 0x7f5ef01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:47902/0 (socket says 172.21.15.64:47902) 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5ef4e7f700 1 -- 172.21.15.64:0/3341763706 learned_addr learned my addr 172.21.15.64:0/3341763706 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eeffff700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ef01021a0 0x7f5ef019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eef7fe700 1 -- 172.21.15.64:0/3341763706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef0198f30 msgr2=0x7f5ef01993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eef7fe700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef0198f30 0x7f5ef01993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:42.996 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eef7fe700 1 -- 172.21.15.64:0/3341763706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ef01021a0 msgr2=0x7f5ef019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eef7fe700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ef01021a0 0x7f5ef019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eef7fe700 1 -- 172.21.15.64:0/3341763706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef01a56b0 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.993+0000 7f5eef7fe700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef019ed20 0x7f5ef01989f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000b550 tx=0x7f5ee000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.994+0000 7f5eed7fa700 1 -- 172.21.15.64:0/3341763706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004230 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.994+0000 7f5eed7fa700 1 -- 172.21.15.64:0/3341763706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee00043d0 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.994+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef01a58e0 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.994+0000 7f5eed7fa700 1 -- 172.21.15.64:0/3341763706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00119e0 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.994+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01a5e20 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.995+0000 7f5eed7fa700 1 -- 172.21.15.64:0/3341763706 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5ee0011bc0 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.995+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0064bb0 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.995+0000 7f5eed7fa700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ed003e9a0 0x7f5ed0040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.995+0000 7f5eed7fa700 1 -- 172.21.15.64:0/3341763706 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ee0052230 con 0x7f5ef019ed20 2024-05-11T12:14:42.997 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.995+0000 7f5eeffff700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ed003e9a0 0x7f5ed0040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:42.998 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:42.995+0000 7f5eeffff700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ed003e9a0 0x7f5ed0040e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004fd0 tx=0x7f5ed801d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:43.005 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.005+0000 7f5eed7fa700 1 -- 172.21.15.64:0/3341763706 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ee0020740 con 0x7f5ef019ed20 2024-05-11T12:14:43.177 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:42 smithi050 ceph-mon[34072]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:43.177 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:43.177 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:43.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:42 smithi047 ceph-mon[30375]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:43.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:43.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:43.314 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.313+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5ef002e570 con 0x7f5ef019ed20 2024-05-11T12:14:43.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:42 smithi064 ceph-mon[34537]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:43.315 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:43.315 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.314+0000 7f5eed7fa700 1 -- 172.21.15.64:0/3341763706 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1277 (secure 0 0 0) 0x7f5ee001fcc0 con 0x7f5ef019ed20 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi064.stdout: 2024-05-11T12:14:43.315 INFO:teuthology.orchestra.run.smithi064.stdout:{"epoch":3,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","modified":"2024-05-11T12:14:36.832606Z","created":"2024-05-11T12:12:33.150058Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:3300","nonce":0},{"type":"v1","addr":"172.21.15.47:6789","nonce":0}]},"addr":"172.21.15.47:6789/0","public_addr":"172.21.15.47:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:3300","nonce":0},{"type":"v1","addr":"172.21.15.64:6789","nonce":0}]},"addr":"172.21.15.64:6789/0","public_addr":"172.21.15.64:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ed003e9a0 msgr2=0x7f5ed0040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ed003e9a0 0x7f5ed0040e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004fd0 tx=0x7f5ed801d040 comp rx=0 tx=0).stop 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef019ed20 msgr2=0x7f5ef01989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef019ed20 0x7f5ef01989f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000b550 tx=0x7f5ee000b860 comp rx=0 tx=0).stop 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 shutdown_connections 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ef0198f30 0x7f5ef01993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ed003e9a0 0x7f5ed0040e60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ef01021a0 0x7f5ef019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 --2- 172.21.15.64:0/3341763706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef019ed20 0x7f5ef01989f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:43.317 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 >> 172.21.15.64:0/3341763706 conn(0x7f5ef00fda30 msgr2=0x7f5ef00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 shutdown_connections 2024-05-11T12:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:43.316+0000 7f5ef68e2700 1 -- 172.21.15.64:0/3341763706 wait complete. 2024-05-11T12:14:43.318 INFO:teuthology.orchestra.run.smithi064.stderr:dumped monmap epoch 3 2024-05-11T12:14:43.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:43 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/3341763706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T12:14:43.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:43.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:43.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:43.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:43.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:43.970 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:43.997 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-11T12:14:43.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph config generate-minimal-conf 2024-05-11T12:14:44.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:43 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/3341763706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T12:14:44.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:44.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:44.249 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:14:44.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:43 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/3341763706' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-11T12:14:44.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:44.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:44.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: Updating smithi047:/etc/ceph/ceph.conf 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: Updating smithi047:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: Updating smithi050:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: Updating smithi064:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.184 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:45.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:45.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:45.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: Updating smithi047:/etc/ceph/ceph.conf 2024-05-11T12:14:45.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: Updating smithi047:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: Updating smithi050:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: Updating smithi064:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:45.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:45.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:45.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: Updating smithi047:/etc/ceph/ceph.conf 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: Updating smithi050:/etc/ceph/ceph.conf 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: Updating smithi064:/etc/ceph/ceph.conf 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: Updating smithi047:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: Updating smithi050:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: Updating smithi064:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/config/ceph.conf 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:45.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:45.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:45.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:46.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:45 smithi047 ceph-mon[30375]: Reconfiguring mon.a (unknown last config time)... 2024-05-11T12:14:46.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:45 smithi047 ceph-mon[30375]: Reconfiguring daemon mon.a on smithi047 2024-05-11T12:14:46.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:45 smithi050 ceph-mon[34072]: Reconfiguring mon.a (unknown last config time)... 2024-05-11T12:14:46.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:45 smithi050 ceph-mon[34072]: Reconfiguring daemon mon.a on smithi047 2024-05-11T12:14:46.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:45 smithi064 ceph-mon[34537]: Reconfiguring mon.a (unknown last config time)... 2024-05-11T12:14:46.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:45 smithi064 ceph-mon[34537]: Reconfiguring daemon mon.a on smithi047 2024-05-11T12:14:46.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:46 smithi047 ceph-mon[30375]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:46.926 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:46.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:46.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:46.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:46.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:46.927 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:46 smithi050 ceph-mon[34072]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.035+0000 7fb91fe37700 1 -- 172.21.15.47:0/63027892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 msgr2=0x7fb9181081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.035+0000 7fb91fe37700 1 --2- 172.21.15.47:0/63027892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb9181081d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb9080099c0 tx=0x7fb908009cd0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.035+0000 7fb91fe37700 1 -- 172.21.15.47:0/63027892 shutdown_connections 2024-05-11T12:14:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.035+0000 7fb91fe37700 1 --2- 172.21.15.47:0/63027892 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb918108900 0x7fb918110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.035+0000 7fb91fe37700 1 --2- 172.21.15.47:0/63027892 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb918103800 0x7fb918103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 --2- 172.21.15.47:0/63027892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb9181081d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 -- 172.21.15.47:0/63027892 >> 172.21.15.47:0/63027892 conn(0x7fb91807a6b0 msgr2=0x7fb91807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 -- 172.21.15.47:0/63027892 shutdown_connections 2024-05-11T12:14:47.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 -- 172.21.15.47:0/63027892 wait complete. 2024-05-11T12:14:47.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 Processor -- start 2024-05-11T12:14:47.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 -- start start 2024-05-11T12:14:47.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb918103800 0x7fb91819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb91819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb918108900 0x7fb918199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181137a0 con 0x7fb9181041b0 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb918113620 con 0x7fb918103800 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.036+0000 7fb91fe37700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb918113920 con 0x7fb918108900 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb91819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb91819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57070/0 (socket says 172.21.15.47:57070) 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 -- 172.21.15.47:0/184427051 learned_addr learned my addr 172.21.15.47:0/184427051 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91dbd3700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb918103800 0x7fb91819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91e3d4700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb918108900 0x7fb918199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 -- 172.21.15.47:0/184427051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb918108900 msgr2=0x7fb918199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb918108900 0x7fb918199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 -- 172.21.15.47:0/184427051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb918103800 msgr2=0x7fb91819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb918103800 0x7fb91819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 -- 172.21.15.47:0/184427051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91819a1e0 con 0x7fb9181041b0 2024-05-11T12:14:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91d3d2700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb91819fe00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b580 tx=0x7fb9080057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb90effd700 1 -- 172.21.15.47:0/184427051 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908027070 con 0x7fb9181041b0 2024-05-11T12:14:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb90effd700 1 -- 172.21.15.47:0/184427051 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb908003fa0 con 0x7fb9181041b0 2024-05-11T12:14:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb90effd700 1 -- 172.21.15.47:0/184427051 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908022d10 con 0x7fb9181041b0 2024-05-11T12:14:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91819a470 con 0x7fb9181041b0 2024-05-11T12:14:47.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.037+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a6790 con 0x7fb9181041b0 2024-05-11T12:14:47.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.038+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb91806a240 con 0x7fb9181041b0 2024-05-11T12:14:47.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.038+0000 7fb90effd700 1 -- 172.21.15.47:0/184427051 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fb90802a070 con 0x7fb9181041b0 2024-05-11T12:14:47.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.039+0000 7fb90effd700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb90403e750 0x7fb904040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:47.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.039+0000 7fb90effd700 1 -- 172.21.15.47:0/184427051 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb908053a60 con 0x7fb9181041b0 2024-05-11T12:14:47.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.039+0000 7fb91dbd3700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb90403e750 0x7fb904040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:47.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.039+0000 7fb91dbd3700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb90403e750 0x7fb904040c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb91819abc0 tx=0x7fb914009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:47.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.044+0000 7fb90effd700 1 -- 172.21.15.47:0/184427051 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb908013070 con 0x7fb9181041b0 2024-05-11T12:14:47.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.257+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb91810bb40 con 0x7fb9181041b0 2024-05-11T12:14:47.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.257+0000 7fb90effd700 1 -- 172.21.15.47:0/184427051 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fb9080095a0 con 0x7fb9181041b0 2024-05-11T12:14:47.257 INFO:teuthology.orchestra.run.smithi047.stdout:# minimal ceph.conf for 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:14:47.257 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-05-11T12:14:47.257 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:14:47.258 INFO:teuthology.orchestra.run.smithi047.stdout: mon_host = [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] 2024-05-11T12:14:47.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb90403e750 msgr2=0x7fb904040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb90403e750 0x7fb904040c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb91819abc0 tx=0x7fb914009190 comp rx=0 tx=0).stop 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 msgr2=0x7fb91819fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb91819fe00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b580 tx=0x7fb9080057c0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 shutdown_connections 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb918108900 0x7fb918199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb90403e750 0x7fb904040c10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb918103800 0x7fb91819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 --2- 172.21.15.47:0/184427051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb9181041b0 0x7fb91819fe00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 >> 172.21.15.47:0/184427051 conn(0x7fb91807a6b0 msgr2=0x7fb9181018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 shutdown_connections 2024-05-11T12:14:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:47.261+0000 7fb91fe37700 1 -- 172.21.15.47:0/184427051 wait complete. 2024-05-11T12:14:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:46 smithi064 ceph-mon[34537]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:47.840 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-11T12:14:47.840 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:14:47.841 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.conf 2024-05-11T12:14:47.869 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:14:47.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:14:47.935 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:14:47.935 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-05-11T12:14:47.962 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:14:47.962 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:14:48.027 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:14:48.027 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.conf 2024-05-11T12:14:48.053 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:14:48.053 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-11T12:14:48.118 INFO:tasks.cephadm:Adding mgr.x on smithi047 2024-05-11T12:14:48.118 INFO:tasks.cephadm:Adding mgr.y on smithi050 2024-05-11T12:14:48.118 INFO:tasks.cephadm:Adding mgr.z on smithi064 2024-05-11T12:14:48.118 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch apply mgr '3;smithi047=x;smithi050=y;smithi064=z' 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: Reconfiguring mon.b (monmap changed)... 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: Reconfiguring daemon mon.b on smithi050 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/184427051' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:48.180 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:48.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: Reconfiguring mon.b (monmap changed)... 2024-05-11T12:14:48.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: Reconfiguring daemon mon.b on smithi050 2024-05-11T12:14:48.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/184427051' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: Reconfiguring mon.b (monmap changed)... 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: Reconfiguring daemon mon.b on smithi050 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/184427051' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-11T12:14:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:48.855 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:14:49.168 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:48 smithi064 ceph-mon[34537]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:49.168 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:48 smithi064 ceph-mon[34537]: Reconfiguring mon.c (monmap changed)... 2024-05-11T12:14:49.168 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:48 smithi064 ceph-mon[34537]: Reconfiguring daemon mon.c on smithi064 2024-05-11T12:14:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:48 smithi050 ceph-mon[34072]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:48 smithi050 ceph-mon[34072]: Reconfiguring mon.c (monmap changed)... 2024-05-11T12:14:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:48 smithi050 ceph-mon[34072]: Reconfiguring daemon mon.c on smithi064 2024-05-11T12:14:49.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:48 smithi047 ceph-mon[30375]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:49.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:48 smithi047 ceph-mon[30375]: Reconfiguring mon.c (monmap changed)... 2024-05-11T12:14:49.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:48 smithi047 ceph-mon[30375]: Reconfiguring daemon mon.c on smithi064 2024-05-11T12:14:50.684 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 -- 172.21.15.64:0/2890078160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc105150 msgr2=0x7f1bdc10ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 --2- 172.21.15.64:0/2890078160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc105150 0x7f1bdc10ba00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009990 tx=0x7f1bcc009ca0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.694 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 -- 172.21.15.64:0/2890078160 shutdown_connections 2024-05-11T12:14:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 --2- 172.21.15.64:0/2890078160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc105150 0x7f1bdc10ba00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 --2- 172.21.15.64:0/2890078160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc1035b0 0x7f1bdc1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 --2- 172.21.15.64:0/2890078160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bdc1047b0 0x7f1bdc104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 -- 172.21.15.64:0/2890078160 >> 172.21.15.64:0/2890078160 conn(0x7f1bdc0feb50 msgr2=0x7f1bdc100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 -- 172.21.15.64:0/2890078160 shutdown_connections 2024-05-11T12:14:50.695 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.684+0000 7f1be44fd700 1 -- 172.21.15.64:0/2890078160 wait complete. 2024-05-11T12:14:50.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 Processor -- start 2024-05-11T12:14:50.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 -- start start 2024-05-11T12:14:50.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bdc1035b0 0x7f1bdc19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc1047b0 0x7f1bdc19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.698 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc105150 0x7f1bdc1a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10f0a0 con 0x7f1bdc1035b0 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10ef20 con 0x7f1bdc105150 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be44fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc10f220 con 0x7f1bdc1047b0 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc1047b0 0x7f1bdc19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc1047b0 0x7f1bdc19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:44596/0 (socket says 172.21.15.64:44596) 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 -- 172.21.15.64:0/3302305687 learned_addr learned my addr 172.21.15.64:0/3302305687 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be2299700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bdc1035b0 0x7f1bdc19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be2a9a700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc105150 0x7f1bdc1a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 -- 172.21.15.64:0/3302305687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc105150 msgr2=0x7f1bdc1a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc105150 0x7f1bdc1a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.699 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 -- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bdc1035b0 msgr2=0x7f1bdc19b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bdc1035b0 0x7f1bdc19b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.685+0000 7f1be1a98700 1 -- 172.21.15.64:0/3302305687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc1a3530 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.686+0000 7f1be1a98700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc1047b0 0x7f1bdc19be10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000d9b0 tx=0x7f1bd000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.686+0000 7f1bdb7fe700 1 -- 172.21.15.64:0/3302305687 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0016070 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.686+0000 7f1bdb7fe700 1 -- 172.21.15.64:0/3302305687 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd0004830 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.686+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc1a3820 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.686+0000 7f1bdb7fe700 1 -- 172.21.15.64:0/3302305687 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0005440 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.686+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc1a3db0 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.687+0000 7f1bdb7fe700 1 -- 172.21.15.64:0/3302305687 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1bd00055e0 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.687+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bdc06a240 con 0x7f1bdc1047b0 2024-05-11T12:14:50.700 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.687+0000 7f1bdb7fe700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1bc403e750 0x7f1bc4040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.687+0000 7f1bdb7fe700 1 -- 172.21.15.64:0/3302305687 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1bd00525a0 con 0x7f1bdc1047b0 2024-05-11T12:14:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.687+0000 7f1be2299700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1bc403e750 0x7f1bc4040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.688+0000 7f1be2299700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1bc403e750 0x7f1bc4040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc104610 tx=0x7f1bd400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:50.701 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.693+0000 7f1bdb7fe700 1 -- 172.21.15.64:0/3302305687 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1bd001f020 con 0x7f1bdc1047b0 2024-05-11T12:14:50.921 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.921+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi050=y;smithi064=z", "target": ["mon-mgr", ""]}) -- 0x7f1bdc076ea0 con 0x7f1bc403e750 2024-05-11T12:14:50.926 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.926+0000 7f1bdb7fe700 1 -- 172.21.15.64:0/3302305687 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1bdc076ea0 con 0x7f1bc403e750 2024-05-11T12:14:50.926 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mgr update... 2024-05-11T12:14:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.928+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1bc403e750 msgr2=0x7f1bc4040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.928+0000 7f1be44fd700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1bc403e750 0x7f1bc4040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc104610 tx=0x7f1bd400b040 comp rx=0 tx=0).stop 2024-05-11T12:14:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.928+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc1047b0 msgr2=0x7f1bdc19be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:50.928 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.928+0000 7f1be44fd700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc1047b0 0x7f1bdc19be10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000d9b0 tx=0x7f1bd000dcc0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 shutdown_connections 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1bc403e750 0x7f1bc4040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1bdc105150 0x7f1bdc1a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bdc1035b0 0x7f1bdc19b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 --2- 172.21.15.64:0/3302305687 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1bdc1047b0 0x7f1bdc19be10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 >> 172.21.15.64:0/3302305687 conn(0x7f1bdc0feb50 msgr2=0x7f1bdc075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 shutdown_connections 2024-05-11T12:14:50.929 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:14:50.929+0000 7f1be44fd700 1 -- 172.21.15.64:0/3302305687 wait complete. 2024-05-11T12:14:51.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:50 smithi050 ceph-mon[34072]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:51.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:50 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:50 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:50 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:51.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:50 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:51.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:50 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:51.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:50 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:50 smithi047 ceph-mon[30375]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:50 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:50 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:50 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:50 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:50 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:50 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:50 smithi064 ceph-mon[34537]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:50 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:50 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:50 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:50 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:50 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:50 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.516 DEBUG:teuthology.orchestra.run.smithi050:mgr.y> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.y.service 2024-05-11T12:14:51.519 DEBUG:teuthology.orchestra.run.smithi064:mgr.z> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.z.service 2024-05-11T12:14:51.559 INFO:tasks.cephadm:Deploying OSDs... 2024-05-11T12:14:51.559 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:14:51.559 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T12:14:51.580 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T12:14:51.581 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-05-11T12:14:51.639 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-05-11 12:13:51.970312243 +0000 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-05-11 12:08:57.373123239 +0000 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-05-11 12:08:57.373123239 +0000 2024-05-11T12:14:51.640 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-05-11 12:08:57.373123239 +0000 2024-05-11T12:14:51.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T12:14:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-05-11T12:14:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-05-11T12:14:51.710 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000128038 s, 4.0 MB/s 2024-05-11T12:14:51.711 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T12:14:51.771 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-05-11T12:14:51.829 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T12:14:51.829 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:51.829 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-11T12:14:51.829 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:51.829 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:51.829 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-05-11 12:13:52.000311600 +0000 2024-05-11T12:14:51.829 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-05-11 12:08:57.719115684 +0000 2024-05-11T12:14:51.830 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-05-11 12:08:57.719115684 +0000 2024-05-11T12:14:51.830 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-05-11 12:08:57.719115684 +0000 2024-05-11T12:14:51.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T12:14:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-05-11T12:14:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-05-11T12:14:51.896 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.00011804 s, 4.3 MB/s 2024-05-11T12:14:51.897 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T12:14:51.956 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-05-11T12:14:51.975 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi050=y;smithi064=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:14:51.975 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: Saving service mgr spec with placement smithi047=x;smithi050=y;smithi064=z;count:3 2024-05-11T12:14:51.975 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:51.976 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:51 smithi064 ceph-mon[34537]: Deploying daemon mgr.z on smithi064 2024-05-11T12:14:52.013 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T12:14:52.014 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:52.014 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-11T12:14:52.014 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:52.014 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:52.014 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-05-11 12:13:52.029310978 +0000 2024-05-11T12:14:52.015 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-05-11 12:08:58.066108107 +0000 2024-05-11T12:14:52.015 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-05-11 12:08:58.066108107 +0000 2024-05-11T12:14:52.015 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-05-11 12:08:58.066108107 +0000 2024-05-11T12:14:52.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi050=y;smithi064=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: Saving service mgr spec with placement smithi047=x;smithi050=y;smithi064=z;count:3 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T12:14:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:52.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:52.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:51 smithi047 ceph-mon[30375]: Deploying daemon mgr.z on smithi064 2024-05-11T12:14:52.051 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-05-11T12:14:52.051 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-05-11T12:14:52.051 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000100954 s, 5.1 MB/s 2024-05-11T12:14:52.053 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T12:14:52.111 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-05-11 12:13:52.058310357 +0000 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-05-11 12:08:58.416100464 +0000 2024-05-11T12:14:52.168 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-05-11 12:08:58.416100464 +0000 2024-05-11T12:14:52.169 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-05-11 12:08:58.416100464 +0000 2024-05-11T12:14:52.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T12:14:52.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi047=x;smithi050=y;smithi064=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:14:52.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: Saving service mgr spec with placement smithi047=x;smithi050=y;smithi064=z;count:3 2024-05-11T12:14:52.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:51 smithi050 ceph-mon[34072]: Deploying daemon mgr.z on smithi064 2024-05-11T12:14:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-05-11T12:14:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-05-11T12:14:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000116825 s, 4.4 MB/s 2024-05-11T12:14:52.235 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T12:14:52.292 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-11T12:14:52.292 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T12:14:52.312 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T12:14:52.313 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-05-11T12:14:52.371 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T12:14:52.371 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:52.371 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-11T12:14:52.371 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:52.372 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:52.372 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-11 12:14:36.218732115 +0000 2024-05-11T12:14:52.372 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-11 12:09:16.949236717 +0000 2024-05-11T12:14:52.372 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-11 12:09:16.949236717 +0000 2024-05-11T12:14:52.372 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-11 12:09:16.949236717 +0000 2024-05-11T12:14:52.372 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T12:14:52.438 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-11T12:14:52.438 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-11T12:14:52.438 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000127263 s, 4.0 MB/s 2024-05-11T12:14:52.440 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T12:14:52.498 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-05-11T12:14:52.555 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T12:14:52.555 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:52.556 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-11T12:14:52.556 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:52.556 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:52.556 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-11 12:14:36.218732115 +0000 2024-05-11T12:14:52.556 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-11 12:09:17.300227646 +0000 2024-05-11T12:14:52.556 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-11 12:09:17.300227646 +0000 2024-05-11T12:14:52.556 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-11 12:09:17.300227646 +0000 2024-05-11T12:14:52.556 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T12:14:52.621 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-11T12:14:52.621 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-11T12:14:52.621 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.00010691 s, 4.8 MB/s 2024-05-11T12:14:52.623 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T12:14:52.682 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-11 12:14:36.219732090 +0000 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-11 12:09:17.654218497 +0000 2024-05-11T12:14:52.740 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-11 12:09:17.654218497 +0000 2024-05-11T12:14:52.741 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-11 12:09:17.654218497 +0000 2024-05-11T12:14:52.741 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T12:14:52.806 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-11T12:14:52.806 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-11T12:14:52.806 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.00012363 s, 4.1 MB/s 2024-05-11T12:14:52.807 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T12:14:52.865 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-05-11T12:14:52.923 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T12:14:52.923 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:52.924 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-11T12:14:52.924 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:52.924 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:52.924 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-11 12:14:36.219732090 +0000 2024-05-11T12:14:52.924 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-11 12:09:17.991209788 +0000 2024-05-11T12:14:52.924 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-11 12:09:17.991209788 +0000 2024-05-11T12:14:52.924 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-11 12:09:17.991209788 +0000 2024-05-11T12:14:52.924 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T12:14:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-11T12:14:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-11T12:14:52.989 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000107976 s, 4.7 MB/s 2024-05-11T12:14:52.991 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T12:14:53.050 DEBUG:teuthology.orchestra.run.smithi064:> set -ex 2024-05-11T12:14:53.050 DEBUG:teuthology.orchestra.run.smithi064:> dd if=/scratch_devs of=/dev/stdout 2024-05-11T12:14:53.067 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-11T12:14:53.067 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_1 2024-05-11T12:14:53.123 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-11T12:14:53.123 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:53.123 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-11T12:14:53.123 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:53.124 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:53.124 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-11 12:14:52.923597586 +0000 2024-05-11T12:14:53.124 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-11 12:09:17.930394501 +0000 2024-05-11T12:14:53.124 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-11 12:09:17.930394501 +0000 2024-05-11T12:14:53.124 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-11 12:09:17.930394501 +0000 2024-05-11T12:14:53.124 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-11T12:14:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-11T12:14:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-11T12:14:53.190 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000267028 s, 1.9 MB/s 2024-05-11T12:14:53.191 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-11T12:14:53.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:52 smithi050 ceph-mon[34072]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:53.234 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:52 smithi064 ceph-mon[34537]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:53.248 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_2 2024-05-11T12:14:53.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:52 smithi047 ceph-mon[30375]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:53.303 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-11T12:14:53.303 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:53.304 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-11T12:14:53.304 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:53.304 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:53.304 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-11 12:14:52.924597566 +0000 2024-05-11T12:14:53.304 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-11 12:09:18.294387042 +0000 2024-05-11T12:14:53.304 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-11 12:09:18.294387042 +0000 2024-05-11T12:14:53.304 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-11 12:09:18.294387042 +0000 2024-05-11T12:14:53.304 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-11T12:14:53.367 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-11T12:14:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-11T12:14:53.368 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000181808 s, 2.8 MB/s 2024-05-11T12:14:53.369 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-11T12:14:53.426 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_3 2024-05-11T12:14:53.481 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-11 12:14:52.924597566 +0000 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-11 12:09:18.778377124 +0000 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-11 12:09:18.778377124 +0000 2024-05-11T12:14:53.482 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-11 12:09:18.778377124 +0000 2024-05-11T12:14:53.482 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-11T12:14:53.548 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-11T12:14:53.548 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-11T12:14:53.548 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000240348 s, 2.1 MB/s 2024-05-11T12:14:53.550 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-11T12:14:53.607 DEBUG:teuthology.orchestra.run.smithi064:> stat /dev/vg_nvme/lv_4 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout:Context: system_u:object_r:device_t:s0 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout:Access: 2024-05-11 12:14:52.925597547 +0000 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout:Modify: 2024-05-11 12:09:19.105370423 +0000 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout:Change: 2024-05-11 12:09:19.105370423 +0000 2024-05-11T12:14:53.664 INFO:teuthology.orchestra.run.smithi064.stdout: Birth: 2024-05-11 12:09:19.105370423 +0000 2024-05-11T12:14:53.664 DEBUG:teuthology.orchestra.run.smithi064:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-11T12:14:53.729 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records in 2024-05-11T12:14:53.729 INFO:teuthology.orchestra.run.smithi064.stderr:1+0 records out 2024-05-11T12:14:53.729 INFO:teuthology.orchestra.run.smithi064.stderr:512 bytes copied, 0.000202918 s, 2.5 MB/s 2024-05-11T12:14:53.731 DEBUG:teuthology.orchestra.run.smithi064:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-11T12:14:53.787 INFO:tasks.cephadm:Deploying osd.0 on smithi047 with /dev/vg_nvme/lv_4... 2024-05-11T12:14:53.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-11T12:14:54.039 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:14:54.613 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:54.428+0000 7f1978afe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T12:14:54.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:54 smithi050 ceph-mon[34072]: Deploying daemon mgr.y on smithi050 2024-05-11T12:14:55.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:54 smithi047 ceph-mon[30375]: Deploying daemon mgr.y on smithi050 2024-05-11T12:14:55.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:55.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-11T12:14:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:54 smithi064 ceph-mon[34537]: Deploying daemon mgr.y on smithi050 2024-05-11T12:14:55.949 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:55 smithi050 systemd[1]: Starting Ceph mgr.y for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:14:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-11T12:14:55.986 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-11T12:14:55.986 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-05-11T12:14:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-05-11T12:14:55.987 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00845292 s, 1.2 GB/s 2024-05-11T12:14:55.987 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-05-11T12:14:56.040 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:55.688+0000 7f1978afe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T12:14:56.040 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:55.864+0000 7f1978afe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T12:14:56.040 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:56.040+0000 7f1978afe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T12:14:56.376 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 podman[35486]: 2024-05-11 12:14:55.981517115 +0000 UTC m=+0.018713752 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:14:56.376 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 podman[35486]: 2024-05-11 12:14:56.08262789 +0000 UTC m=+0.119824553 container create 70f85aab87bfd2d62eebac49caa8872d5a2ac79ff33d50c8acbb5da4782bfce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:14:56.376 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 podman[35486]: 2024-05-11 12:14:56.293139351 +0000 UTC m=+0.330335972 container init 70f85aab87bfd2d62eebac49caa8872d5a2ac79ff33d50c8acbb5da4782bfce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:14:56.376 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 podman[35486]: 2024-05-11 12:14:56.295771346 +0000 UTC m=+0.332967976 container start 70f85aab87bfd2d62eebac49caa8872d5a2ac79ff33d50c8acbb5da4782bfce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:14:56.376 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 ceph-mgr[35505]: -- 172.21.15.50:0/2183190377 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5592c82c45a0 con 0x5592c8298c00 2024-05-11T12:14:56.376 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 bash[35486]: 70f85aab87bfd2d62eebac49caa8872d5a2ac79ff33d50c8acbb5da4782bfce4 2024-05-11T12:14:56.613 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:56.338+0000 7f1978afe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T12:14:56.613 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:56.433+0000 7f1978afe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T12:14:56.646 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_4 2024-05-11T12:14:56.655 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 systemd[1]: Started Ceph mgr.y for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:14:56.656 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:56.461+0000 7f6d78dfe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-11T12:14:56.892 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:14:56.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:56 smithi050 ceph-mon[34072]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:56.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:56.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:56.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:56.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:56.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:56.949 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:56.656+0000 7f6d78dfe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:14:56.949 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:56.764+0000 7f6d78dfe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-11T12:14:57.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:56 smithi047 ceph-mon[30375]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:57.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:57.113 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:56.643+0000 7f1978afe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T12:14:57.113 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:56.847+0000 7f1978afe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:14:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:56 smithi064 ceph-mon[34537]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:14:57.448 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:57.098+0000 7f6d78dfe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-11T12:14:57.448 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:57.193+0000 7f6d78dfe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-11T12:14:57.739 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:57.505+0000 7f1978afe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T12:14:57.739 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:57.739+0000 7f1978afe200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T12:14:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 -- 172.21.15.47:0/669451143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 msgr2=0x7fda6c10b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 --2- 172.21.15.47:0/669451143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c10b710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda64009f90 tx=0x7fda6400b330 comp rx=0 tx=0).stop 2024-05-11T12:14:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 -- 172.21.15.47:0/669451143 shutdown_connections 2024-05-11T12:14:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 --2- 172.21.15.47:0/669451143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c10b710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 --2- 172.21.15.47:0/669451143 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda6c104040 0x7fda6c106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 --2- 172.21.15.47:0/669451143 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda6c106970 0x7fda6c108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 -- 172.21.15.47:0/669451143 >> 172.21.15.47:0/669451143 conn(0x7fda6c0fda50 msgr2=0x7fda6c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:14:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 -- 172.21.15.47:0/669451143 shutdown_connections 2024-05-11T12:14:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.530+0000 7fda70a34700 1 -- 172.21.15.47:0/669451143 wait complete. 2024-05-11T12:14:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 Processor -- start 2024-05-11T12:14:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 -- start start 2024-05-11T12:14:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda6c104040 0x7fda6c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda6c106970 0x7fda6c199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10f070 con 0x7fda6c1092a0 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10eef0 con 0x7fda6c104040 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda70a34700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c10f1f0 con 0x7fda6c106970 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda6ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda69d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda6c106970 0x7fda6c199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda6a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda6c104040 0x7fda6c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda6ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c1a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34084/0 (socket says 172.21.15.47:34084) 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda6ad9d700 1 -- 172.21.15.47:0/95122882 learned_addr learned my addr 172.21.15.47:0/95122882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.531+0000 7fda6a59c700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda6c104040 0x7fda6c1996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:41978/0 (socket says 172.21.15.47:41978) 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda69d9b700 1 -- 172.21.15.47:0/95122882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda6c104040 msgr2=0x7fda6c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda69d9b700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda6c104040 0x7fda6c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda69d9b700 1 -- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 msgr2=0x7fda6c1a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda69d9b700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c1a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:14:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda69d9b700 1 -- 172.21.15.47:0/95122882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c1a1340 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda6ad9d700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda69d9b700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda6c106970 0x7fda6c199c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fda6000ebc0 tx=0x7fda6000eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6000cc90 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c101e90 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda600044d0 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60010430 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.532+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c102420 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.533+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fda600105d0 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.533+0000 7fda5b7fe700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fda5403e750 0x7fda54040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.533+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c06a240 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.534+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fda60052b80 con 0x7fda6c106970 2024-05-11T12:14:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.534+0000 7fda6a59c700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fda5403e750 0x7fda54040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:14:58.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.534+0000 7fda6a59c700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fda5403e750 0x7fda54040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009c60 tx=0x7fda5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:14:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.539+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fda600203a0 con 0x7fda6c106970 2024-05-11T12:14:58.613 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:58.192+0000 7f1978afe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T12:14:58.614 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:58.292+0000 7f1978afe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T12:14:58.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:14:58.766+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fda6c063990 con 0x7fda5403e750 2024-05-11T12:14:58.850 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:58.496+0000 7f6d78dfe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-11T12:14:58.851 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:58.672+0000 7f6d78dfe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-11T12:14:58.851 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:58.850+0000 7f6d78dfe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-11T12:14:58.973 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:58.676+0000 7f1978afe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:14:59.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:14:59.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:59.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.151 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:14:59.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.363 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:59.048+0000 7f1978afe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:59.143+0000 7f1978afe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:14:59.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:14:59.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:14:59.448 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:59.151+0000 7f6d78dfe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-11T12:14:59.448 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:59.248+0000 7f6d78dfe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-11T12:14:59.863 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:59.399+0000 7f1978afe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T12:14:59.864 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:59.537+0000 7f1978afe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T12:14:59.864 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:59.635+0000 7f1978afe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T12:14:59.948 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:59.459+0000 7f6d78dfe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-11T12:14:59.948 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:14:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:14:59.660+0000 7f6d78dfe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-11T12:15:00.087 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: Reconfiguring mgr.x (unknown last config time)... 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: Reconfiguring daemon mgr.x on smithi047 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:00.088 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:14:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: Reconfiguring mgr.x (unknown last config time)... 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: Reconfiguring daemon mgr.x on smithi047 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:00.293 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:14:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.363 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:14:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:14:59.974+0000 7f1978afe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T12:15:00.363 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:15:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:15:00.072+0000 7f1978afe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: Reconfiguring mgr.x (unknown last config time)... 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: Reconfiguring daemon mgr.x on smithi047 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: from='client.24124 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:14:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:00.698 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:00.293+0000 7f6d78dfe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-11T12:15:00.698 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:00.521+0000 7f6d78dfe200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-11T12:15:00.978 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:15:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:15:00.701+0000 7f1978afe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T12:15:00.978 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:15:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z[36468]: 2024-05-11T12:15:00.926+0000 7f1978afe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T12:15:00.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:00.985+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7fda60021440 con 0x7fda6c106970 2024-05-11T12:15:01.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:00 smithi047 ceph-mon[30375]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:01.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:00 smithi047 ceph-mon[30375]: Standby manager daemon z started 2024-05-11T12:15:01.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:00 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T12:15:01.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:00 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T12:15:01.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:00 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T12:15:01.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:00 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T12:15:01.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:00 smithi064 ceph-mon[34537]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:00 smithi064 ceph-mon[34537]: Standby manager daemon z started 2024-05-11T12:15:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:00 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T12:15:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:00 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T12:15:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:00 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T12:15:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:00 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T12:15:01.443 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:00 smithi050 ceph-mon[34072]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:00 smithi050 ceph-mon[34072]: Standby manager daemon z started 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:00 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:00 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:00 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:00 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.64:0/2042743378' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:00.959+0000 7f6d78dfe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:01.061+0000 7f6d78dfe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-11T12:15:01.444 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:01.443+0000 7f6d78dfe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-11T12:15:02.172 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:01.817+0000 7f6d78dfe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:01.915+0000 7f6d78dfe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:01 smithi050 ceph-mon[34072]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:01 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/231226279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f335b8d2-f243-4dbe-862c-98d13f68228f"}]: dispatch 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:01 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/231226279' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f335b8d2-f243-4dbe-862c-98d13f68228f"}]': finished 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:01 smithi050 ceph-mon[34072]: osdmap e5: 1 total, 0 up, 1 in 2024-05-11T12:15:02.173 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:01 smithi047 ceph-mon[30375]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-11T12:15:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T12:15:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:01 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/231226279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f335b8d2-f243-4dbe-862c-98d13f68228f"}]: dispatch 2024-05-11T12:15:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:01 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/231226279' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f335b8d2-f243-4dbe-862c-98d13f68228f"}]': finished 2024-05-11T12:15:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:01 smithi047 ceph-mon[30375]: osdmap e5: 1 total, 0 up, 1 in 2024-05-11T12:15:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:02.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:01 smithi064 ceph-mon[34537]: mgrmap e14: x(active, since 79s), standbys: z 2024-05-11T12:15:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-11T12:15:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:01 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/231226279' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f335b8d2-f243-4dbe-862c-98d13f68228f"}]: dispatch 2024-05-11T12:15:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:01 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/231226279' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f335b8d2-f243-4dbe-862c-98d13f68228f"}]': finished 2024-05-11T12:15:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:01 smithi064 ceph-mon[34537]: osdmap e5: 1 total, 0 up, 1 in 2024-05-11T12:15:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:02.448 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:02.172+0000 7f6d78dfe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-11T12:15:02.449 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:02.312+0000 7f6d78dfe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-11T12:15:02.449 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:02.411+0000 7f6d78dfe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-11T12:15:03.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:02 smithi050 ceph-mon[34072]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:03.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:02 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1713084220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:03.199 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:02.753+0000 7f6d78dfe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-11T12:15:03.199 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:02.848+0000 7f6d78dfe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-11T12:15:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:02 smithi047 ceph-mon[30375]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:02 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1713084220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:03.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:02 smithi064 ceph-mon[34537]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:03.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:02 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1713084220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:03.930 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:03.480+0000 7f6d78dfe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-11T12:15:03.930 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:15:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y[35501]: 2024-05-11T12:15:03.704+0000 7f6d78dfe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-11T12:15:04.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:04.003+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mon.1 v2:172.21.15.64:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fda6001f340 con 0x7fda6c106970 2024-05-11T12:15:04.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:04 smithi050 ceph-mon[34072]: Standby manager daemon y started 2024-05-11T12:15:04.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:04 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T12:15:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:04 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T12:15:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:04 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T12:15:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:04 smithi050 ceph-mon[34072]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T12:15:04.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:03 smithi047 ceph-mon[30375]: Standby manager daemon y started 2024-05-11T12:15:04.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:03 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T12:15:04.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:03 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T12:15:04.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:03 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T12:15:04.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:03 smithi047 ceph-mon[30375]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T12:15:04.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:04 smithi064 ceph-mon[34537]: Standby manager daemon y started 2024-05-11T12:15:04.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:04 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-11T12:15:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:04 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-11T12:15:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:04 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-11T12:15:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:04 smithi064 ceph-mon[34537]: from='mgr.? 172.21.15.50:0/2230553937' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-11T12:15:05.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:05 smithi064 ceph-mon[34537]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:05 smithi064 ceph-mon[34537]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-05-11T12:15:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:05 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T12:15:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:05 smithi047 ceph-mon[30375]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:05 smithi047 ceph-mon[30375]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-05-11T12:15:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:05 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T12:15:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:05 smithi050 ceph-mon[34072]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:05 smithi050 ceph-mon[34072]: mgrmap e15: x(active, since 82s), standbys: z, y 2024-05-11T12:15:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:05 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-11T12:15:06.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:06 smithi047 ceph-mon[30375]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:06.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:06 smithi064 ceph-mon[34537]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:06.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:06 smithi050 ceph-mon[34072]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:08.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:08 smithi047 ceph-mon[30375]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:08.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:08 smithi064 ceph-mon[34537]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:08.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:08 smithi050 ceph-mon[34072]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:09.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:09 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-11T12:15:09.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:09 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:09.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:09 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-11T12:15:09.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:09 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:09.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:09 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-11T12:15:09.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:09 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:10.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:10 smithi047 ceph-mon[30375]: Deploying daemon osd.0 on smithi047 2024-05-11T12:15:10.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:10 smithi047 ceph-mon[30375]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:10 smithi064 ceph-mon[34537]: Deploying daemon osd.0 on smithi047 2024-05-11T12:15:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:10 smithi064 ceph-mon[34537]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:10.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:10 smithi050 ceph-mon[34072]: Deploying daemon osd.0 on smithi047 2024-05-11T12:15:10.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:10 smithi050 ceph-mon[34072]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:12.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:12 smithi047 ceph-mon[30375]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:12.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:12 smithi064 ceph-mon[34537]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:12.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:12 smithi050 ceph-mon[34072]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:14.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:14 smithi047 ceph-mon[30375]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:14.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:14 smithi064 ceph-mon[34537]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:14.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:14 smithi050 ceph-mon[34072]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:16.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:15 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:16.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:15 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:16.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:15 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:16.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:15 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:16.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:15 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:16.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:15 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:16.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:15 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:16.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:15 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:16.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:15 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:17.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:16 smithi047 ceph-mon[30375]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:17.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:16 smithi064 ceph-mon[34537]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:17.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:16 smithi050 ceph-mon[34072]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:18 smithi047 ceph-mon[30375]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:18.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.630+0000 7fda5b7fe700 1 -- 172.21.15.47:0/95122882 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fda6c063990 con 0x7fda5403e750 2024-05-11T12:15:18.630 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 0 on host 'smithi047' 2024-05-11T12:15:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.633+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fda5403e750 msgr2=0x7fda54040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.633+0000 7fda70a34700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fda5403e750 0x7fda54040c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fda5c009c60 tx=0x7fda5c009260 comp rx=0 tx=0).stop 2024-05-11T12:15:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.633+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda6c106970 msgr2=0x7fda6c199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:18.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.633+0000 7fda70a34700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda6c106970 0x7fda6c199c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fda6000ebc0 tx=0x7fda6000eed0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 shutdown_connections 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c1092a0 0x7fda6c1a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fda5403e750 0x7fda54040c10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fda6c104040 0x7fda6c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 --2- 172.21.15.47:0/95122882 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fda6c106970 0x7fda6c199c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 >> 172.21.15.47:0/95122882 conn(0x7fda6c0fda50 msgr2=0x7fda6c104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 shutdown_connections 2024-05-11T12:15:18.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:15:18.634+0000 7fda70a34700 1 -- 172.21.15.47:0/95122882 wait complete. 2024-05-11T12:15:18.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:18.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:18.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:18 smithi050 ceph-mon[34072]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:18.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:18.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:18.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:18.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:18 smithi064 ceph-mon[34537]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:19.193 DEBUG:teuthology.orchestra.run.smithi047:osd.0> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.0.service 2024-05-11T12:15:19.196 INFO:tasks.cephadm:Deploying osd.1 on smithi050 with /dev/vg_nvme/lv_4... 2024-05-11T12:15:19.196 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-11T12:15:19.451 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:15:19.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:19 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:19.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:19 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:19.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:19 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:19.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:19 smithi047 ceph-mon[30375]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-11T12:15:19.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:19 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:19.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:19 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:19.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:19 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:19.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:19 smithi064 ceph-mon[34537]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-11T12:15:19.930 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:19 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:19.931 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:19 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:19.931 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:19 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:19.931 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:19 smithi050 ceph-mon[34072]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-11T12:15:20.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:20 smithi047 ceph-mon[30375]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:20.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:20 smithi047 ceph-mon[30375]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-11T12:15:20.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:20 smithi047 ceph-mon[30375]: osdmap e6: 1 total, 0 up, 1 in 2024-05-11T12:15:20.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:20 smithi047 ceph-mon[30375]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:15:20.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:20.740 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:15:20 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0[42685]: 2024-05-11T12:15:20.602+0000 7ff50f78c700 -1 osd.0 0 waiting for initial osdmap 2024-05-11T12:15:20.740 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:15:20 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0[42685]: 2024-05-11T12:15:20.605+0000 7ff50b57f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:15:20.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:20 smithi064 ceph-mon[34537]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:20.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:20 smithi064 ceph-mon[34537]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-11T12:15:20.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:20 smithi064 ceph-mon[34537]: osdmap e6: 1 total, 0 up, 1 in 2024-05-11T12:15:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:20 smithi064 ceph-mon[34537]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:15:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:20.873 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:20 smithi050 ceph-mon[34072]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:20.873 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:20 smithi050 ceph-mon[34072]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-11T12:15:20.873 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:20 smithi050 ceph-mon[34072]: osdmap e6: 1 total, 0 up, 1 in 2024-05-11T12:15:20.873 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:20 smithi050 ceph-mon[34072]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:15:20.873 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:21.455 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-11T12:15:21.455 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-11T12:15:21.455 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-11T12:15:21.455 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-11T12:15:21.455 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00888258 s, 1.2 GB/s 2024-05-11T12:15:21.455 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-11T12:15:21.798 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:21 smithi050 ceph-mon[34072]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:15:21.798 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:21 smithi050 ceph-mon[34072]: osdmap e7: 1 total, 0 up, 1 in 2024-05-11T12:15:21.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:21 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:21.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:21 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:21.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:21 smithi064 ceph-mon[34537]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:15:21.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:21 smithi064 ceph-mon[34537]: osdmap e7: 1 total, 0 up, 1 in 2024-05-11T12:15:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:21 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:21 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:21.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:21 smithi047 ceph-mon[30375]: from='osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:15:21.941 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:21 smithi047 ceph-mon[30375]: osdmap e7: 1 total, 0 up, 1 in 2024-05-11T12:15:21.942 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:21.942 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:22.085 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-05-11T12:15:22.334 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] boot 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: osdmap e8: 1 total, 1 up, 1 in 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:22.710 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:22.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:15:22.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:22.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:22.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:22.711 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:22.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] boot 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: osdmap e8: 1 total, 1 up, 1 in 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: osd.0 [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] boot 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: osdmap e8: 1 total, 1 up, 1 in 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:23.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:23.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:23.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 -- 172.21.15.50:0/3414555574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 msgr2=0x7fbb3c108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 --2- 172.21.15.50:0/3414555574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c108d70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009990 tx=0x7fbb2c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 -- 172.21.15.50:0/3414555574 shutdown_connections 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 --2- 172.21.15.50:0/3414555574 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb3c1092b0 0x7fbb3c10b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 --2- 172.21.15.50:0/3414555574 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c104050 0x7fbb3c106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 --2- 172.21.15.50:0/3414555574 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c108d70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 -- 172.21.15.50:0/3414555574 >> 172.21.15.50:0/3414555574 conn(0x7fbb3c0fda80 msgr2=0x7fbb3c0ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 -- 172.21.15.50:0/3414555574 shutdown_connections 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 -- 172.21.15.50:0/3414555574 wait complete. 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.831+0000 7fbb43962700 1 Processor -- start 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb43962700 1 -- start start 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb43962700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c104050 0x7fbb3c103960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb43962700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb43962700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb3c1092b0 0x7fbb3c102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb43962700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c10f040 con 0x7fbb3c1092b0 2024-05-11T12:15:23.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb43962700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c10eec0 con 0x7fbb3c106980 2024-05-11T12:15:23.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb43962700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3c10f1c0 con 0x7fbb3c104050 2024-05-11T12:15:23.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43096/0 (socket says 172.21.15.50:43096) 2024-05-11T12:15:23.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 -- 172.21.15.50:0/3332898998 learned_addr learned my addr 172.21.15.50:0/3332898998 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-11T12:15:23.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb416fe700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c104050 0x7fbb3c103960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb41eff700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb3c1092b0 0x7fbb3c102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 -- 172.21.15.50:0/3332898998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c104050 msgr2=0x7fbb3c103960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c104050 0x7fbb3c103960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 -- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb3c1092b0 msgr2=0x7fbb3c102540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb3c1092b0 0x7fbb3c102540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 -- 172.21.15.50:0/3332898998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb3c102e10 con 0x7fbb3c106980 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb41eff700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb3c1092b0 0x7fbb3c102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.832+0000 7fbb40efd700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c102000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c006010 tx=0x7fbb2c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.833+0000 7fbb327fc700 1 -- 172.21.15.50:0/3332898998 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c027070 con 0x7fbb3c106980 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.833+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3c1a3800 con 0x7fbb3c106980 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.833+0000 7fbb327fc700 1 -- 172.21.15.50:0/3332898998 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb2c022470 con 0x7fbb3c106980 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.833+0000 7fbb327fc700 1 -- 172.21.15.50:0/3332898998 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00f6c0 con 0x7fbb3c106980 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.833+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3c1a3d00 con 0x7fbb3c106980 2024-05-11T12:15:23.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.834+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb3c06a240 con 0x7fbb3c106980 2024-05-11T12:15:23.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.835+0000 7fbb327fc700 1 -- 172.21.15.50:0/3332898998 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fbb2c02a070 con 0x7fbb3c106980 2024-05-11T12:15:23.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.836+0000 7fbb327fc700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbb280b2870 0x7fbb280b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:23.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.836+0000 7fbb327fc700 1 -- 172.21.15.50:0/3332898998 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fbb2c0e16f0 con 0x7fbb3c106980 2024-05-11T12:15:23.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.836+0000 7fbb416fe700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbb280b2870 0x7fbb280b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:23.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.836+0000 7fbb416fe700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbb280b2870 0x7fbb280b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005ea0 tx=0x7fbb38005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:23.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:23.840+0000 7fbb327fc700 1 -- 172.21.15.50:0/3332898998 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb2c013070 con 0x7fbb3c106980 2024-05-11T12:15:23.840 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:23 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi047 2024-05-11T12:15:23.840 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:23 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi047 to 17093M 2024-05-11T12:15:23.840 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:23 smithi050 ceph-mon[34072]: osdmap e9: 1 total, 1 up, 1 in 2024-05-11T12:15:24.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:23 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi047 2024-05-11T12:15:24.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:23 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi047 to 17093M 2024-05-11T12:15:24.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:23 smithi047 ceph-mon[30375]: osdmap e9: 1 total, 1 up, 1 in 2024-05-11T12:15:24.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:24.065+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbb3c063990 con 0x7fbb280b2870 2024-05-11T12:15:24.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:23 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi047 2024-05-11T12:15:24.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:23 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi047 to 17093M 2024-05-11T12:15:24.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:23 smithi064 ceph-mon[34537]: osdmap e9: 1 total, 1 up, 1 in 2024-05-11T12:15:24.935 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:24 smithi050 ceph-mon[34072]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:24.935 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:15:24.935 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:15:24.935 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:24 smithi047 ceph-mon[30375]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:15:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:15:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:25.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:24 smithi064 ceph-mon[34537]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:25.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:15:25.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:15:25.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:25.904 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:25 smithi050 ceph-mon[34072]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:26.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:25 smithi047 ceph-mon[30375]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:26.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:25 smithi064 ceph-mon[34537]: from='client.24155 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:26 smithi050 ceph-mon[34072]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:27.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:26 smithi047 ceph-mon[30375]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:27.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:26 smithi064 ceph-mon[34537]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:27.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:27 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/4009134341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]: dispatch 2024-05-11T12:15:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:27 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]: dispatch 2024-05-11T12:15:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:27 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]': finished 2024-05-11T12:15:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:27 smithi050 ceph-mon[34072]: osdmap e10: 2 total, 1 up, 2 in 2024-05-11T12:15:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:27 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/106294015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:27 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/4009134341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]: dispatch 2024-05-11T12:15:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:27 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]: dispatch 2024-05-11T12:15:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:27 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]': finished 2024-05-11T12:15:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:27 smithi047 ceph-mon[30375]: osdmap e10: 2 total, 1 up, 2 in 2024-05-11T12:15:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:27 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/106294015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:28.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:27 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/4009134341' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]: dispatch 2024-05-11T12:15:28.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:27 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]: dispatch 2024-05-11T12:15:28.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:27 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1bacadec-81bc-4e48-b2f7-67645202bba0"}]': finished 2024-05-11T12:15:28.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:27 smithi064 ceph-mon[34537]: osdmap e10: 2 total, 1 up, 2 in 2024-05-11T12:15:28.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:28.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:27 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/106294015' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:29.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:28 smithi050 ceph-mon[34072]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:28 smithi047 ceph-mon[30375]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:29.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:28 smithi064 ceph-mon[34537]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:31.148 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:30 smithi050 ceph-mon[34072]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:31.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:30 smithi047 ceph-mon[30375]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:31.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:30 smithi064 ceph-mon[34537]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:32.982 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:32 smithi050 ceph-mon[34072]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:33.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:32 smithi047 ceph-mon[30375]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:33.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:32 smithi064 ceph-mon[34537]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:34.923 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:34 smithi050 ceph-mon[34072]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:34.923 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-11T12:15:34.923 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:35.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:34 smithi047 ceph-mon[30375]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:35.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-11T12:15:35.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:35.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:34 smithi064 ceph-mon[34537]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-11T12:15:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:36.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:35 smithi050 ceph-mon[34072]: Deploying daemon osd.1 on smithi050 2024-05-11T12:15:36.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:35 smithi047 ceph-mon[30375]: Deploying daemon osd.1 on smithi050 2024-05-11T12:15:36.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:35 smithi064 ceph-mon[34537]: Deploying daemon osd.1 on smithi050 2024-05-11T12:15:37.157 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:36 smithi050 ceph-mon[34072]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:36 smithi047 ceph-mon[30375]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:37.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:36 smithi064 ceph-mon[34537]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:39.157 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:38 smithi050 ceph-mon[34072]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:39.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:38 smithi047 ceph-mon[30375]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:39.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:38 smithi064 ceph-mon[34537]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:40.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:40 smithi047 ceph-mon[30375]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:40.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:40.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:40.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:40.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:40 smithi064 ceph-mon[34537]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:40.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:40.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:40.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:40.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:40 smithi050 ceph-mon[34072]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:40.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:42.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:42 smithi047 ceph-mon[30375]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:42.783 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 1 on host 'smithi050' 2024-05-11T12:15:42.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.783+0000 7fbb327fc700 1 -- 172.21.15.50:0/3332898998 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbb3c063990 con 0x7fbb280b2870 2024-05-11T12:15:42.784 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:42 smithi050 ceph-mon[34072]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:42.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbb280b2870 msgr2=0x7fbb280b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:42.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbb280b2870 0x7fbb280b4d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb38005ea0 tx=0x7fbb38005e30 comp rx=0 tx=0).stop 2024-05-11T12:15:42.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 msgr2=0x7fbb3c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:42.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c102000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c006010 tx=0x7fbb2c0047b0 comp rx=0 tx=0).stop 2024-05-11T12:15:42.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 shutdown_connections 2024-05-11T12:15:42.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbb3c1092b0 0x7fbb3c102540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbb3c104050 0x7fbb3c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbb280b2870 0x7fbb280b4d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 --2- 172.21.15.50:0/3332898998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbb3c106980 0x7fbb3c102000 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 >> 172.21.15.50:0/3332898998 conn(0x7fbb3c0fda80 msgr2=0x7fbb3c104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 shutdown_connections 2024-05-11T12:15:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:15:42.786+0000 7fbb43962700 1 -- 172.21.15.50:0/3332898998 wait complete. 2024-05-11T12:15:42.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:42 smithi064 ceph-mon[34537]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:43.427 DEBUG:teuthology.orchestra.run.smithi050:osd.1> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.1.service 2024-05-11T12:15:43.429 INFO:tasks.cephadm:Deploying osd.2 on smithi064 with /dev/vg_nvme/lv_4... 2024-05-11T12:15:43.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-11T12:15:43.683 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:15:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:43 smithi047 ceph-mon[30375]: from='osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T12:15:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:43 smithi047 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T12:15:44.053 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.053 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.053 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.053 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.053 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:43 smithi064 ceph-mon[34537]: from='osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T12:15:44.053 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:43 smithi064 ceph-mon[34537]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T12:15:44.104 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.104 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.104 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.104 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:44.104 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:43 smithi050 ceph-mon[34072]: from='osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T12:15:44.104 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:43 smithi050 ceph-mon[34072]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-11T12:15:45.101 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:44 smithi064 ceph-mon[34537]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:45.101 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:44 smithi064 ceph-mon[34537]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-11T12:15:45.101 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:44 smithi064 ceph-mon[34537]: osdmap e11: 2 total, 1 up, 2 in 2024-05-11T12:15:45.101 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:45.101 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:44 smithi064 ceph-mon[34537]: from='osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:15:45.101 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:44 smithi064 ceph-mon[34537]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:15:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:44 smithi050 ceph-mon[34072]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:44 smithi050 ceph-mon[34072]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-11T12:15:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:44 smithi050 ceph-mon[34072]: osdmap e11: 2 total, 1 up, 2 in 2024-05-11T12:15:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:44 smithi050 ceph-mon[34072]: from='osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:15:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:44 smithi050 ceph-mon[34072]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:15:45.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:44 smithi047 ceph-mon[30375]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:45.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:44 smithi047 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-11T12:15:45.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:44 smithi047 ceph-mon[30375]: osdmap e11: 2 total, 1 up, 2 in 2024-05-11T12:15:45.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:45.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:44 smithi047 ceph-mon[30375]: from='osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:15:45.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:44 smithi047 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:15:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-11T12:15:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-11T12:15:45.582 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-11T12:15:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-11T12:15:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114247 s, 918 MB/s 2024-05-11T12:15:45.582 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-11T12:15:45.694 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:15:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1[38703]: 2024-05-11T12:15:45.338+0000 7f442a135700 -1 osd.1 0 waiting for initial osdmap 2024-05-11T12:15:45.694 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:15:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1[38703]: 2024-05-11T12:15:45.343+0000 7f4421f20700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:15:45.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:45 smithi050 ceph-mon[34072]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:15:45.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:45 smithi050 ceph-mon[34072]: osdmap e12: 2 total, 1 up, 2 in 2024-05-11T12:15:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:45 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:45 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:45 smithi050 ceph-mon[34072]: from='osd.1 ' entity='osd.1' 2024-05-11T12:15:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:45 smithi064 ceph-mon[34537]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:15:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:45 smithi064 ceph-mon[34537]: osdmap e12: 2 total, 1 up, 2 in 2024-05-11T12:15:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:45 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:45 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:45 smithi064 ceph-mon[34537]: from='osd.1 ' entity='osd.1' 2024-05-11T12:15:46.232 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_4 2024-05-11T12:15:46.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:45 smithi047 ceph-mon[30375]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:15:46.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:45 smithi047 ceph-mon[30375]: osdmap e12: 2 total, 1 up, 2 in 2024-05-11T12:15:46.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:45 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:46.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:45 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:46.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:45 smithi047 ceph-mon[30375]: from='osd.1 ' entity='osd.1' 2024-05-11T12:15:46.477 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:15:46.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] boot 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: osdmap e13: 2 total, 2 up, 2 in 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] boot 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: osdmap e13: 2 total, 2 up, 2 in 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:47.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:15:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:15:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-11T12:15:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: osd.1 [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] boot 2024-05-11T12:15:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: osdmap e13: 2 total, 2 up, 2 in 2024-05-11T12:15:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:15:47.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:15:48.111 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:47 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi050 2024-05-11T12:15:48.112 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:47 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi050 to 17093M 2024-05-11T12:15:48.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 -- 172.21.15.64:0/3156724784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d781092a0 msgr2=0x7f8d7810b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:48.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 --2- 172.21.15.64:0/3156724784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d781092a0 0x7f8d7810b710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000b0a0 tx=0x7f8d7000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:15:48.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 -- 172.21.15.64:0/3156724784 shutdown_connections 2024-05-11T12:15:48.140 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 --2- 172.21.15.64:0/3156724784 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d781092a0 0x7f8d7810b710 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 --2- 172.21.15.64:0/3156724784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d78104040 0x7f8d78106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 --2- 172.21.15.64:0/3156724784 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d78106970 0x7f8d78108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 -- 172.21.15.64:0/3156724784 >> 172.21.15.64:0/3156724784 conn(0x7f8d780fda50 msgr2=0x7f8d780ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 -- 172.21.15.64:0/3156724784 shutdown_connections 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.111+0000 7f8d7da57700 1 -- 172.21.15.64:0/3156724784 wait complete. 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 Processor -- start 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 -- start start 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d78104040 0x7f8d78199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d78106970 0x7f8d78199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:48.141 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d781092a0 0x7f8d781a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7810f030 con 0x7f8d78106970 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7810eeb0 con 0x7f8d781092a0 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d7da57700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7810f1b0 con 0x7f8d78104040 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d78104040 0x7f8d78199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d76ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d78104040 0x7f8d78199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:50054/0 (socket says 172.21.15.64:50054) 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.112+0000 7f8d76ffd700 1 -- 172.21.15.64:0/1969010920 learned_addr learned my addr 172.21.15.64:0/1969010920 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d777fe700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d781092a0 0x7f8d781a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d767fc700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d78106970 0x7f8d78199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d76ffd700 1 -- 172.21.15.64:0/1969010920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d781092a0 msgr2=0x7f8d781a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d76ffd700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d781092a0 0x7f8d781a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:48.142 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d76ffd700 1 -- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d78106970 msgr2=0x7f8d78199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d76ffd700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d78106970 0x7f8d78199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d76ffd700 1 -- 172.21.15.64:0/1969010920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d781a1390 con 0x7f8d78104040 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d767fc700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d78106970 0x7f8d78199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d76ffd700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d78104040 0x7f8d78199730 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e9d0 tx=0x7f8d6800ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d7ca55700 1 -- 172.21.15.64:0/1969010920 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800cbd0 con 0x7f8d78104040 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d7ca55700 1 -- 172.21.15.64:0/1969010920 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d6800cd70 con 0x7f8d78104040 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d781a1680 con 0x7f8d78104040 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d7ca55700 1 -- 172.21.15.64:0/1969010920 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68018710 con 0x7f8d78104040 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.113+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d781a1c10 con 0x7f8d78104040 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.114+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d7806a240 con 0x7f8d78104040 2024-05-11T12:15:48.143 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.115+0000 7f8d7ca55700 1 -- 172.21.15.64:0/1969010920 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f8d68010430 con 0x7f8d78104040 2024-05-11T12:15:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.116+0000 7f8d7ca55700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f8d600b2870 0x7f8d600b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:15:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.116+0000 7f8d7ca55700 1 -- 172.21.15.64:0/1969010920 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f8d680e0bc0 con 0x7f8d78104040 2024-05-11T12:15:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.116+0000 7f8d767fc700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f8d600b2870 0x7f8d600b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:15:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.117+0000 7f8d767fc700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f8d600b2870 0x7f8d600b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009be0 tx=0x7f8d6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:15:48.144 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.121+0000 7f8d7ca55700 1 -- 172.21.15.64:0/1969010920 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8d680aa130 con 0x7f8d78104040 2024-05-11T12:15:48.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:47 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi050 2024-05-11T12:15:48.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:47 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi050 to 17093M 2024-05-11T12:15:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:47 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi050 2024-05-11T12:15:48.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:47 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi050 to 17093M 2024-05-11T12:15:48.344 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:15:48.344+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8d78063990 con 0x7f8d600b2870 2024-05-11T12:15:49.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:48 smithi064 ceph-mon[34537]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:48 smithi064 ceph-mon[34537]: osdmap e14: 2 total, 2 up, 2 in 2024-05-11T12:15:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:15:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:15:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:48 smithi050 ceph-mon[34072]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:48 smithi050 ceph-mon[34072]: osdmap e14: 2 total, 2 up, 2 in 2024-05-11T12:15:49.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:15:49.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:15:49.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:49.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:48 smithi047 ceph-mon[30375]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:49.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:48 smithi047 ceph-mon[30375]: osdmap e14: 2 total, 2 up, 2 in 2024-05-11T12:15:49.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:15:49.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:15:49.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:49.906 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:49 smithi064 ceph-mon[34537]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:50.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:49 smithi050 ceph-mon[34072]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:50.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:49 smithi047 ceph-mon[30375]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:15:51.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:50 smithi064 ceph-mon[34537]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:51.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:50 smithi050 ceph-mon[34072]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:50 smithi047 ceph-mon[30375]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:52.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:51 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/411200431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]: dispatch 2024-05-11T12:15:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:51 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]: dispatch 2024-05-11T12:15:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:51 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]': finished 2024-05-11T12:15:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:51 smithi064 ceph-mon[34537]: osdmap e15: 3 total, 2 up, 3 in 2024-05-11T12:15:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:15:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:51 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/3757724110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:52.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:51 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/411200431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]: dispatch 2024-05-11T12:15:52.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:51 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]: dispatch 2024-05-11T12:15:52.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:51 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]': finished 2024-05-11T12:15:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:51 smithi050 ceph-mon[34072]: osdmap e15: 3 total, 2 up, 3 in 2024-05-11T12:15:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:15:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:51 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/3757724110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:51 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/411200431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]: dispatch 2024-05-11T12:15:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:51 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]: dispatch 2024-05-11T12:15:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:51 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e"}]': finished 2024-05-11T12:15:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:51 smithi047 ceph-mon[30375]: osdmap e15: 3 total, 2 up, 3 in 2024-05-11T12:15:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:15:52.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:51 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/3757724110' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:15:53.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:52 smithi064 ceph-mon[34537]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:53.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:52 smithi050 ceph-mon[34072]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:53.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:52 smithi047 ceph-mon[30375]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:55.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:54 smithi064 ceph-mon[34537]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:55.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:54 smithi050 ceph-mon[34072]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:54 smithi047 ceph-mon[30375]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:57.035 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:56 smithi064 ceph-mon[34537]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:57.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:56 smithi050 ceph-mon[34072]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:57.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:56 smithi047 ceph-mon[30375]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:59.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:58 smithi064 ceph-mon[34537]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-11T12:15:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:59.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:58 smithi050 ceph-mon[34072]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:59.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-11T12:15:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:58 smithi047 ceph-mon[30375]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:15:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-11T12:15:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:15:59.993 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:15:59 smithi064 ceph-mon[34537]: Deploying daemon osd.2 on smithi064 2024-05-11T12:16:00.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:15:59 smithi050 ceph-mon[34072]: Deploying daemon osd.2 on smithi064 2024-05-11T12:16:00.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:15:59 smithi047 ceph-mon[30375]: Deploying daemon osd.2 on smithi064 2024-05-11T12:16:00.895 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:00 smithi064 ceph-mon[34537]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:01.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:00 smithi050 ceph-mon[34072]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:00 smithi047 ceph-mon[30375]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:03.037 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:02 smithi064 ceph-mon[34537]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:03.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:02 smithi050 ceph-mon[34072]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:02 smithi047 ceph-mon[30375]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:04 smithi050 ceph-mon[34072]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:04 smithi047 ceph-mon[30375]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:05.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:04 smithi064 ceph-mon[34537]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:06.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:06 smithi050 ceph-mon[34072]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:06 smithi047 ceph-mon[30375]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:07.063 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:06 smithi064 ceph-mon[34537]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:07.063 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:07.063 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:07.063 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:07.063 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:07.063 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:07.367 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 2 on host 'smithi064' 2024-05-11T12:16:07.368 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.367+0000 7f8d7ca55700 1 -- 172.21.15.64:0/1969010920 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8d78063990 con 0x7f8d600b2870 2024-05-11T12:16:07.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f8d600b2870 msgr2=0x7f8d600b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:07.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f8d600b2870 0x7f8d600b4d30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009be0 tx=0x7f8d6c009260 comp rx=0 tx=0).stop 2024-05-11T12:16:07.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d78104040 msgr2=0x7f8d78199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:07.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d78104040 0x7f8d78199730 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800e9d0 tx=0x7f8d6800ed90 comp rx=0 tx=0).stop 2024-05-11T12:16:07.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 shutdown_connections 2024-05-11T12:16:07.370 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f8d600b2870 0x7f8d600b4d30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:07.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8d781092a0 0x7f8d781a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:07.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8d78104040 0x7f8d78199730 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:07.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 --2- 172.21.15.64:0/1969010920 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8d78106970 0x7f8d78199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:07.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 >> 172.21.15.64:0/1969010920 conn(0x7f8d780fda50 msgr2=0x7f8d78104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:16:07.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 shutdown_connections 2024-05-11T12:16:07.371 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:16:07.370+0000 7f8d7da57700 1 -- 172.21.15.64:0/1969010920 wait complete. 2024-05-11T12:16:08.020 DEBUG:teuthology.orchestra.run.smithi064:osd.2> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.2.service 2024-05-11T12:16:08.022 INFO:tasks.cephadm:Deploying osd.3 on smithi047 with /dev/vg_nvme/lv_3... 2024-05-11T12:16:08.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-11T12:16:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:16:08.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:08.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:08.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:08.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:08 smithi064 ceph-mon[34537]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:08.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:08 smithi064 ceph-mon[34537]: from='osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T12:16:08.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:08 smithi064 ceph-mon[34537]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T12:16:08.634 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:08.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:08.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:08.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:08 smithi047 ceph-mon[30375]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:08.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:08 smithi047 ceph-mon[30375]: from='osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T12:16:08.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:08 smithi047 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T12:16:08.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:08.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:08.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:08.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:08 smithi050 ceph-mon[34072]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:08.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:08 smithi050 ceph-mon[34072]: from='osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T12:16:08.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:08 smithi050 ceph-mon[34072]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-11T12:16:09.423 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:09 smithi064 ceph-mon[34537]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-11T12:16:09.423 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:09 smithi064 ceph-mon[34537]: osdmap e16: 3 total, 2 up, 3 in 2024-05-11T12:16:09.423 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:09 smithi064 ceph-mon[34537]: from='osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:16:09.423 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:09 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:09.423 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:09 smithi064 ceph-mon[34537]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:16:09.423 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2[39870]: 2024-05-11T12:16:09.372+0000 7f12eb371700 -1 osd.2 0 waiting for initial osdmap 2024-05-11T12:16:09.423 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2[39870]: 2024-05-11T12:16:09.377+0000 7f12e7164700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:16:09.621 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:09 smithi047 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-11T12:16:09.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:09 smithi047 ceph-mon[30375]: osdmap e16: 3 total, 2 up, 3 in 2024-05-11T12:16:09.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:09 smithi047 ceph-mon[30375]: from='osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:16:09.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:09 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:09.622 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:09 smithi047 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:16:09.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:09 smithi050 ceph-mon[34072]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-11T12:16:09.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:09 smithi050 ceph-mon[34072]: osdmap e16: 3 total, 2 up, 3 in 2024-05-11T12:16:09.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:09 smithi050 ceph-mon[34072]: from='osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:16:09.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:09 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:09.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:09 smithi050 ceph-mon[34072]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:16:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-11T12:16:10.345 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-11T12:16:10.346 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-05-11T12:16:10.346 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-05-11T12:16:10.346 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00841114 s, 1.2 GB/s 2024-05-11T12:16:10.346 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-05-11T12:16:10.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:10 smithi064 ceph-mon[34537]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:16:10.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:10 smithi064 ceph-mon[34537]: osdmap e17: 3 total, 2 up, 3 in 2024-05-11T12:16:10.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:10 smithi064 ceph-mon[34537]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:10.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:10 smithi047 ceph-mon[30375]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:16:10.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:10 smithi047 ceph-mon[30375]: osdmap e17: 3 total, 2 up, 3 in 2024-05-11T12:16:10.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:10.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:10.660 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:10 smithi047 ceph-mon[30375]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:10.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:10 smithi050 ceph-mon[34072]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:16:10.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:10 smithi050 ceph-mon[34072]: osdmap e17: 3 total, 2 up, 3 in 2024-05-11T12:16:10.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:10.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:10.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:10 smithi050 ceph-mon[34072]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-11T12:16:11.053 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_3 2024-05-11T12:16:11.313 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:16:11.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:16:11.669 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] boot 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: osdmap e18: 3 total, 3 up, 3 in 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:11.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] boot 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: osdmap e18: 3 total, 3 up, 3 in 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:11.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: osd.2 [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] boot 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: osdmap e18: 3 total, 3 up, 3 in 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:11.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:12.677 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:12 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi064 2024-05-11T12:16:12.677 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:12 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-11T12:16:12.677 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:12 smithi050 ceph-mon[34072]: osdmap e19: 3 total, 3 up, 3 in 2024-05-11T12:16:12.677 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:12 smithi050 ceph-mon[34072]: pgmap v84: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:12.677 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:12 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:16:12.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:12 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi064 2024-05-11T12:16:12.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:12 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-11T12:16:12.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:12 smithi047 ceph-mon[30375]: osdmap e19: 3 total, 3 up, 3 in 2024-05-11T12:16:12.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:12 smithi047 ceph-mon[30375]: pgmap v84: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:12.738 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:12 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:16:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:12 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi064 2024-05-11T12:16:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:12 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi064 to 17088M 2024-05-11T12:16:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:12 smithi064 ceph-mon[34537]: osdmap e19: 3 total, 3 up, 3 in 2024-05-11T12:16:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:12 smithi064 ceph-mon[34537]: pgmap v84: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:12 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:16:12.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 -- 172.21.15.47:0/739529209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b01027b0 msgr2=0x7fa2b0102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/739529209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b01027b0 0x7fa2b0102c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009990 tx=0x7fa2a0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 -- 172.21.15.47:0/739529209 shutdown_connections 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/739529209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2b0103170 0x7fa2b01035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/739529209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b0101e50 0x7fa2b0102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/739529209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b01027b0 0x7fa2b0102c30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 -- 172.21.15.47:0/739529209 >> 172.21.15.47:0/739529209 conn(0x7fa2b00fda70 msgr2=0x7fa2b00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 -- 172.21.15.47:0/739529209 shutdown_connections 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.984+0000 7fa2b73bf700 1 -- 172.21.15.47:0/739529209 wait complete. 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 Processor -- start 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 -- start start 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b0101e50 0x7fa2b019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2b01027b0 0x7fa2b019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:12.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b0103170 0x7fa2b01a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01112a0 con 0x7fa2b0101e50 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0111120 con 0x7fa2b01027b0 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b73bf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0111420 con 0x7fa2b0103170 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b515b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b0101e50 0x7fa2b019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b515b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b0101e50 0x7fa2b019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50172/0 (socket says 172.21.15.47:50172) 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.985+0000 7fa2b515b700 1 -- 172.21.15.47:0/4090963295 learned_addr learned my addr 172.21.15.47:0/4090963295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b495a700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2b01027b0 0x7fa2b019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b595c700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b0103170 0x7fa2b01a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b515b700 1 -- 172.21.15.47:0/4090963295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b0103170 msgr2=0x7fa2b01a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b515b700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b0103170 0x7fa2b01a2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b515b700 1 -- 172.21.15.47:0/4090963295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2b01027b0 msgr2=0x7fa2b019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b515b700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2b01027b0 0x7fa2b019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b515b700 1 -- 172.21.15.47:0/4090963295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b01a3510 con 0x7fa2b0101e50 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b595c700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b0103170 0x7fa2b01a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b495a700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2b01027b0 0x7fa2b019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:16:12.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b515b700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b0101e50 0x7fa2b019b8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00d9b0 tx=0x7fa2ac00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:16:12.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac017070 con 0x7fa2b0101e50 2024-05-11T12:16:12.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2ac004830 con 0x7fa2b0101e50 2024-05-11T12:16:12.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac0053a0 con 0x7fa2b0101e50 2024-05-11T12:16:12.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b01a3800 con 0x7fa2b0101e50 2024-05-11T12:16:12.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.986+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b01a3cf0 con 0x7fa2b0101e50 2024-05-11T12:16:12.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.987+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b006a240 con 0x7fa2b0101e50 2024-05-11T12:16:12.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.989+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fa2ac0049e0 con 0x7fa2b0101e50 2024-05-11T12:16:12.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.990+0000 7fa2a67fc700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa29c0b28f0 0x7fa29c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:12.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.990+0000 7fa2b495a700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa29c0b28f0 0x7fa29c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:12.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.990+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fa2ac0e10e0 con 0x7fa2b0101e50 2024-05-11T12:16:12.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.990+0000 7fa2b495a700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa29c0b28f0 0x7fa29c0b4db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000b5c0 tx=0x7fa2a0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:16:12.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:12.994+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa2ac020020 con 0x7fa2b0101e50 2024-05-11T12:16:13.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:13.216+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa2b0063990 con 0x7fa29c0b28f0 2024-05-11T12:16:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-11T12:16:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 ceph-mon[30375]: osdmap e20: 3 total, 3 up, 3 in 2024-05-11T12:16:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:16:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:16:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:16:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:13.525 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47387]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T12:16:13.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47387]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47387]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47387]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:13.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47390]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T12:16:13.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47390]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47390]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:13.684 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-11T12:16:13.684 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 ceph-mon[34537]: osdmap e20: 3 total, 3 up, 3 in 2024-05-11T12:16:13.684 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:16:13.684 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:16:13.685 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:16:13.685 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:13.685 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43507]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T12:16:13.685 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43507]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.685 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43507]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.685 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43507]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:13.685 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43510]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T12:16:13.686 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43510]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.686 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43510]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.686 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43510]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:13.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-11T12:16:13.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 ceph-mon[34072]: osdmap e20: 3 total, 3 up, 3 in 2024-05-11T12:16:13.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:16:13.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:16:13.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:16:13.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42816]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42816]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42816]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42816]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42819]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42819]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42819]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.699 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42819]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:13.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47393]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T12:16:13.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47393]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47393]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47393]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:13.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47396]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T12:16:13.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47396]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:13.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47396]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:13.896 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:13 smithi047 sudo[47396]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43513]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43513]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43513]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43513]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43516]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43516]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43516]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:13 smithi064 sudo[43516]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42822]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42822]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42822]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42822]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42825]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-11T12:16:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:13 smithi050 sudo[42825]: pam_unix(sudo:session): session closed for user root 2024-05-11T12:16:14.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:14.410+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa2ac0a5e30 con 0x7fa2b0101e50 2024-05-11T12:16:14.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:16:14.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-11T12:16:14.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: osdmap e21: 3 total, 3 up, 3 in 2024-05-11T12:16:14.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:14 smithi047 ceph-mon[30375]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: osdmap e21: 3 total, 3 up, 3 in 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:14 smithi050 ceph-mon[34072]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: osdmap e21: 3 total, 3 up, 3 in 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-11T12:16:14.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:14 smithi064 ceph-mon[34537]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-11T12:16:15.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:15 smithi047 ceph-mon[30375]: osdmap e22: 3 total, 3 up, 3 in 2024-05-11T12:16:15.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:15 smithi047 ceph-mon[30375]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-11T12:16:15.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:15 smithi050 ceph-mon[34072]: osdmap e22: 3 total, 3 up, 3 in 2024-05-11T12:16:15.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:15 smithi050 ceph-mon[34072]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-11T12:16:15.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:15 smithi064 ceph-mon[34537]: osdmap e22: 3 total, 3 up, 3 in 2024-05-11T12:16:15.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:15 smithi064 ceph-mon[34537]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-11T12:16:16.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:16 smithi050 ceph-mon[34072]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:16.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:16 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2039035123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71e86e-e235-47c9-bb5e-ebe25a5f6900"}]: dispatch 2024-05-11T12:16:16.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:16 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2039035123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da71e86e-e235-47c9-bb5e-ebe25a5f6900"}]': finished 2024-05-11T12:16:16.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:16 smithi050 ceph-mon[34072]: osdmap e23: 4 total, 3 up, 4 in 2024-05-11T12:16:16.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:16 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:16.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:16 smithi047 ceph-mon[30375]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:16.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:16 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2039035123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71e86e-e235-47c9-bb5e-ebe25a5f6900"}]: dispatch 2024-05-11T12:16:16.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:16 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2039035123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da71e86e-e235-47c9-bb5e-ebe25a5f6900"}]': finished 2024-05-11T12:16:16.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:16 smithi047 ceph-mon[30375]: osdmap e23: 4 total, 3 up, 4 in 2024-05-11T12:16:16.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:16 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:16.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:16 smithi064 ceph-mon[34537]: pgmap v89: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:16 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2039035123' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da71e86e-e235-47c9-bb5e-ebe25a5f6900"}]: dispatch 2024-05-11T12:16:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:16 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2039035123' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da71e86e-e235-47c9-bb5e-ebe25a5f6900"}]': finished 2024-05-11T12:16:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:16 smithi064 ceph-mon[34537]: osdmap e23: 4 total, 3 up, 4 in 2024-05-11T12:16:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:16 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:17.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:17 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/97385095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:16:17.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:17 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/97385095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:16:17.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:17 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/97385095' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:16:18.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:18 smithi050 ceph-mon[34072]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:18.779 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:18 smithi047 ceph-mon[30375]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:18.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:18 smithi064 ceph-mon[34537]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:20.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:20 smithi047 ceph-mon[30375]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:20.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:20 smithi064 ceph-mon[34537]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:20.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:20 smithi050 ceph-mon[34072]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:22.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:22 smithi064 ceph-mon[34537]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:22.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:22 smithi050 ceph-mon[34072]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:22 smithi047 ceph-mon[30375]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:23.850 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-11T12:16:23.850 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:23.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-11T12:16:23.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:24.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-11T12:16:24.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:24.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:24 smithi047 ceph-mon[30375]: Deploying daemon osd.3 on smithi047 2024-05-11T12:16:24.699 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:24 smithi047 ceph-mon[30375]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:24.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:24 smithi064 ceph-mon[34537]: Deploying daemon osd.3 on smithi047 2024-05-11T12:16:24.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:24 smithi064 ceph-mon[34537]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:24.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:24 smithi050 ceph-mon[34072]: Deploying daemon osd.3 on smithi047 2024-05-11T12:16:24.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:24 smithi050 ceph-mon[34072]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:26.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:26 smithi047 ceph-mon[30375]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:26.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:26 smithi064 ceph-mon[34537]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:26.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:26 smithi050 ceph-mon[34072]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:28.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:28 smithi047 ceph-mon[30375]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:28.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:28 smithi064 ceph-mon[34537]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:28 smithi050 ceph-mon[34072]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:30.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:30.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:30.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:30.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:30.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:30.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:31.128 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:30 smithi047 ceph-mon[30375]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:31.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:30 smithi064 ceph-mon[34537]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:31.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:30 smithi050 ceph-mon[34072]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:32.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.776+0000 7fa2a67fc700 1 -- 172.21.15.47:0/4090963295 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2b0063990 con 0x7fa29c0b28f0 2024-05-11T12:16:32.778 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 3 on host 'smithi047' 2024-05-11T12:16:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.778+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa29c0b28f0 msgr2=0x7fa29c0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.778+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa29c0b28f0 0x7fa29c0b4db0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa2a000b5c0 tx=0x7fa2a0005fb0 comp rx=0 tx=0).stop 2024-05-11T12:16:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.778+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b0101e50 msgr2=0x7fa2b019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.778+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b0101e50 0x7fa2b019b8b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00d9b0 tx=0x7fa2ac00dd70 comp rx=0 tx=0).stop 2024-05-11T12:16:32.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 shutdown_connections 2024-05-11T12:16:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa2b0103170 0x7fa2b01a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa2b0101e50 0x7fa2b019b8b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa29c0b28f0 0x7fa29c0b4db0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 --2- 172.21.15.47:0/4090963295 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa2b01027b0 0x7fa2b019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 >> 172.21.15.47:0/4090963295 conn(0x7fa2b00fda70 msgr2=0x7fa2b00ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:16:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 shutdown_connections 2024-05-11T12:16:32.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:16:32.779+0000 7fa2b73bf700 1 -- 172.21.15.47:0/4090963295 wait complete. 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:33.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:33.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:33.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:33.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:33.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:33.487 DEBUG:teuthology.orchestra.run.smithi047:osd.3> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.3.service 2024-05-11T12:16:33.489 INFO:tasks.cephadm:Deploying osd.4 on smithi050 with /dev/vg_nvme/lv_3... 2024-05-11T12:16:33.490 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-11T12:16:33.747 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:16:34.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:33 smithi064 ceph-mon[34537]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-11T12:16:34.158 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:33 smithi050 ceph-mon[34072]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-11T12:16:34.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:33 smithi047 ceph-mon[30375]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-11T12:16:34.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:34 smithi047 ceph-mon[30375]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:34.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:34 smithi047 ceph-mon[30375]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-11T12:16:34.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:34 smithi047 ceph-mon[30375]: osdmap e24: 4 total, 3 up, 4 in 2024-05-11T12:16:34.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:34 smithi047 ceph-mon[30375]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:16:34.986 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:34.987 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:16:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:16:34.861+0000 7f4fe28e0700 -1 osd.3 0 waiting for initial osdmap 2024-05-11T12:16:34.987 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:16:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:16:34.866+0000 7f4fde6d3700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:16:35.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:34 smithi064 ceph-mon[34537]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:35.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:34 smithi064 ceph-mon[34537]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-11T12:16:35.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:34 smithi064 ceph-mon[34537]: osdmap e24: 4 total, 3 up, 4 in 2024-05-11T12:16:35.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:34 smithi064 ceph-mon[34537]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:16:35.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:35.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:34 smithi050 ceph-mon[34072]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:35.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:34 smithi050 ceph-mon[34072]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-11T12:16:35.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:34 smithi050 ceph-mon[34072]: osdmap e24: 4 total, 3 up, 4 in 2024-05-11T12:16:35.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:34 smithi050 ceph-mon[34072]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:16:35.150 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:35.735 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-11T12:16:35.735 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-11T12:16:35.735 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-11T12:16:35.735 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-11T12:16:35.736 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00847391 s, 1.2 GB/s 2024-05-11T12:16:35.736 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-11T12:16:36.051 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:35 smithi050 ceph-mon[34072]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:16:36.052 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:35 smithi050 ceph-mon[34072]: osdmap e25: 4 total, 3 up, 4 in 2024-05-11T12:16:36.052 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:35 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.052 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:35 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.224 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:35 smithi047 ceph-mon[30375]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:16:36.224 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:35 smithi047 ceph-mon[30375]: osdmap e25: 4 total, 3 up, 4 in 2024-05-11T12:16:36.224 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:35 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.224 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:35 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:35 smithi064 ceph-mon[34537]: from='osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:16:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:35 smithi064 ceph-mon[34537]: osdmap e25: 4 total, 3 up, 4 in 2024-05-11T12:16:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:35 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:35 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.365 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-05-11T12:16:36.619 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:16:36.905 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:16:36.905 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:16:36.905 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:36.905 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.905 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] boot 2024-05-11T12:16:36.905 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: osdmap e26: 4 total, 4 up, 4 in 2024-05-11T12:16:36.905 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: osdmap e27: 4 total, 4 up, 4 in 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:36.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] boot 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: osdmap e26: 4 total, 4 up, 4 in 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: osdmap e27: 4 total, 4 up, 4 in 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:37.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:37.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:37.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:37.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: osd.3 [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] boot 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: osdmap e26: 4 total, 4 up, 4 in 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: osdmap e27: 4 total, 4 up, 4 in 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:16:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:38.106 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:37 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi047 2024-05-11T12:16:38.106 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:37 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi047 to 8546M 2024-05-11T12:16:38.106 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:37 smithi050 ceph-mon[34072]: osdmap e28: 4 total, 4 up, 4 in 2024-05-11T12:16:38.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:37 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi047 2024-05-11T12:16:38.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:37 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi047 to 8546M 2024-05-11T12:16:38.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:37 smithi047 ceph-mon[30375]: osdmap e28: 4 total, 4 up, 4 in 2024-05-11T12:16:38.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 -- 172.21.15.50:0/3349946053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 msgr2=0x7f1608104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:38.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 --2- 172.21.15.50:0/3349946053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608104c10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f16040099c0 tx=0x7f1604009cd0 comp rx=0 tx=0).stop 2024-05-11T12:16:38.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 -- 172.21.15.50:0/3349946053 shutdown_connections 2024-05-11T12:16:38.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 --2- 172.21.15.50:0/3349946053 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1608105150 0x7f160810ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:38.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 --2- 172.21.15.50:0/3349946053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16081035b0 0x7f16081039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 --2- 172.21.15.50:0/3349946053 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608104c10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 -- 172.21.15.50:0/3349946053 >> 172.21.15.50:0/3349946053 conn(0x7f16080feb50 msgr2=0x7f1608100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 -- 172.21.15.50:0/3349946053 shutdown_connections 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.351+0000 7f160feaf700 1 -- 172.21.15.50:0/3349946053 wait complete. 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 Processor -- start 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 -- start start 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16081035b0 0x7f1608199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1608105150 0x7f16081a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f160810f140 con 0x7f16081035b0 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f160810efc0 con 0x7f16081047b0 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160feaf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f160810f2c0 con 0x7f1608105150 2024-05-11T12:16:38.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160d44a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160d44a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:42560/0 (socket says 172.21.15.50:42560) 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160d44a700 1 -- 172.21.15.50:0/813821191 learned_addr learned my addr 172.21.15.50:0/813821191 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.352+0000 7f160dc4b700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16081035b0 0x7f1608199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160e44c700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1608105150 0x7f16081a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160d44a700 1 -- 172.21.15.50:0/813821191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1608105150 msgr2=0x7f16081a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160d44a700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1608105150 0x7f16081a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160d44a700 1 -- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16081035b0 msgr2=0x7f1608199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160d44a700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16081035b0 0x7f1608199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160d44a700 1 -- 172.21.15.50:0/813821191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16081a1470 con 0x7f16081047b0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160d44a700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608199d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1604003790 tx=0x7f1604003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f15feffd700 1 -- 172.21.15.50:0/813821191 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604027070 con 0x7f16081047b0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16081a1700 con 0x7f16081047b0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f15feffd700 1 -- 172.21.15.50:0/813821191 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1604003f80 con 0x7f16081047b0 2024-05-11T12:16:38.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f15feffd700 1 -- 172.21.15.50:0/813821191 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160400f460 con 0x7f16081047b0 2024-05-11T12:16:38.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.353+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16081a1cb0 con 0x7f16081047b0 2024-05-11T12:16:38.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.354+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160806a240 con 0x7f16081047b0 2024-05-11T12:16:38.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.355+0000 7f15feffd700 1 -- 172.21.15.50:0/813821191 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f160402a070 con 0x7f16081047b0 2024-05-11T12:16:38.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.356+0000 7f15feffd700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15f40b2880 0x7f15f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:16:38.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.356+0000 7f15feffd700 1 -- 172.21.15.50:0/813821191 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7f16040e27d0 con 0x7f16081047b0 2024-05-11T12:16:38.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.356+0000 7f160dc4b700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15f40b2880 0x7f15f40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:16:38.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.357+0000 7f160dc4b700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15f40b2880 0x7f15f40b4d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15f8006fd0 tx=0x7f15f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:16:38.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.362+0000 7f15feffd700 1 -- 172.21.15.50:0/813821191 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1604031070 con 0x7f16081047b0 2024-05-11T12:16:38.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:37 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi047 2024-05-11T12:16:38.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:37 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi047 to 8546M 2024-05-11T12:16:38.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:37 smithi064 ceph-mon[34537]: osdmap e28: 4 total, 4 up, 4 in 2024-05-11T12:16:38.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:38.584+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1608063990 con 0x7f15f40b2880 2024-05-11T12:16:39.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:38 smithi050 ceph-mon[34072]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:39.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:16:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:16:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:39.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:38 smithi047 ceph-mon[30375]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:39.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:16:39.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:16:39.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:39.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:38 smithi064 ceph-mon[34537]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:39.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:16:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:16:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:40.152 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:39 smithi050 ceph-mon[34072]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:16:40.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:39 smithi047 ceph-mon[30375]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:16:40.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:39 smithi064 ceph-mon[34537]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:16:41.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:40 smithi050 ceph-mon[34072]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:41.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:40 smithi047 ceph-mon[30375]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:41.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:40 smithi064 ceph-mon[34537]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:42.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:41 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/1244641312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]: dispatch 2024-05-11T12:16:42.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:41 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]: dispatch 2024-05-11T12:16:42.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:41 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]': finished 2024-05-11T12:16:42.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:41 smithi047 ceph-mon[30375]: osdmap e29: 5 total, 4 up, 5 in 2024-05-11T12:16:42.176 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:16:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:41 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/1244641312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]: dispatch 2024-05-11T12:16:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:41 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]: dispatch 2024-05-11T12:16:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:41 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]': finished 2024-05-11T12:16:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:41 smithi050 ceph-mon[34072]: osdmap e29: 5 total, 4 up, 5 in 2024-05-11T12:16:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:16:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:41 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/1244641312' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]: dispatch 2024-05-11T12:16:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:41 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]: dispatch 2024-05-11T12:16:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:41 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a94dfdef-2e27-4f70-b821-19770c72d177"}]': finished 2024-05-11T12:16:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:41 smithi064 ceph-mon[34537]: osdmap e29: 5 total, 4 up, 5 in 2024-05-11T12:16:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:16:43.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:42 smithi050 ceph-mon[34072]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:43.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:42 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/3709248030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:16:43.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:42 smithi047 ceph-mon[30375]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:43.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:42 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/3709248030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:16:43.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:42 smithi064 ceph-mon[34537]: pgmap v109: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:43.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:42 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/3709248030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:16:45.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:44 smithi050 ceph-mon[34072]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-11T12:16:45.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:44 smithi047 ceph-mon[30375]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-11T12:16:45.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:44 smithi064 ceph-mon[34537]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-05-11T12:16:47.112 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:46 smithi050 ceph-mon[34072]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-11T12:16:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:46 smithi047 ceph-mon[30375]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-11T12:16:47.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:46 smithi064 ceph-mon[34537]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-11T12:16:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:48 smithi050 ceph-mon[34072]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:16:49.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-11T12:16:49.389 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:49.390 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:48 smithi064 ceph-mon[34537]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:16:49.390 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-11T12:16:49.390 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:49.390 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:48 smithi047 ceph-mon[30375]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:16:49.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-11T12:16:49.391 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:50.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:49 smithi050 ceph-mon[34072]: Deploying daemon osd.4 on smithi050 2024-05-11T12:16:50.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:49 smithi047 ceph-mon[30375]: Deploying daemon osd.4 on smithi050 2024-05-11T12:16:50.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:49 smithi064 ceph-mon[34537]: Deploying daemon osd.4 on smithi050 2024-05-11T12:16:51.172 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:50 smithi050 ceph-mon[34072]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:16:51.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:50 smithi047 ceph-mon[30375]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:16:51.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:50 smithi064 ceph-mon[34537]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:16:53.038 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:52 smithi050 ceph-mon[34072]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-11T12:16:53.166 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:52 smithi064 ceph-mon[34537]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-11T12:16:53.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:52 smithi047 ceph-mon[30375]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-11T12:16:54.941 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:54 smithi050 ceph-mon[34072]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-11T12:16:55.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:54 smithi047 ceph-mon[30375]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-11T12:16:55.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:54 smithi064 ceph-mon[34537]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-11T12:16:56.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:55 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:56.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:55 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:56.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:55 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:56.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:55 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:56.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:55 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:56.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:55 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:56.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:55 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:56.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:55 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:16:56.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:55 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:57.058 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:56 smithi050 ceph-mon[34072]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:57.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:56 smithi047 ceph-mon[30375]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:57.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:56 smithi064 ceph-mon[34537]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:59.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.005+0000 7f15feffd700 1 -- 172.21.15.50:0/813821191 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1608063990 con 0x7f15f40b2880 2024-05-11T12:16:59.005 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 4 on host 'smithi050' 2024-05-11T12:16:59.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.007+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15f40b2880 msgr2=0x7f15f40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.007+0000 7f160feaf700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15f40b2880 0x7f15f40b4d40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f15f8006fd0 tx=0x7f15f8008040 comp rx=0 tx=0).stop 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.007+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 msgr2=0x7f1608199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.007+0000 7f160feaf700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608199d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1604003790 tx=0x7f1604003a10 comp rx=0 tx=0).stop 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 shutdown_connections 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1608105150 0x7f16081a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16081035b0 0x7f1608199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15f40b2880 0x7f15f40b4d40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 --2- 172.21.15.50:0/813821191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f16081047b0 0x7f1608199d50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 >> 172.21.15.50:0/813821191 conn(0x7f16080feb50 msgr2=0x7f1608107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 shutdown_connections 2024-05-11T12:16:59.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:16:59.008+0000 7f160feaf700 1 -- 172.21.15.50:0/813821191 wait complete. 2024-05-11T12:16:59.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: from='osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T12:16:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:58 smithi050 ceph-mon[34072]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: from='osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T12:16:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:58 smithi047 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T12:16:59.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:16:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:16:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:16:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:16:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: from='osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T12:16:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:58 smithi064 ceph-mon[34537]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-11T12:16:59.646 DEBUG:teuthology.orchestra.run.smithi050:osd.4> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.4.service 2024-05-11T12:16:59.648 INFO:tasks.cephadm:Deploying osd.5 on smithi064 with /dev/vg_nvme/lv_3... 2024-05-11T12:16:59.648 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-11T12:16:59.906 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:17:00.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:00.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:00.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:00.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-11T12:17:00.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: osdmap e30: 5 total, 4 up, 5 in 2024-05-11T12:17:00.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:00.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: from='osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:17:00.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:16:59 smithi047 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: osdmap e30: 5 total, 4 up, 5 in 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: from='osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:17:00.285 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:16:59 smithi064 ceph-mon[34537]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:17:00.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:00.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:00.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:00.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-11T12:17:00.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: osdmap e30: 5 total, 4 up, 5 in 2024-05-11T12:17:00.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:00.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: from='osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:17:00.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:16:59 smithi050 ceph-mon[34072]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:17:00.835 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:17:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4[45670]: 2024-05-11T12:17:00.510+0000 7f39d98a1700 -1 osd.4 0 waiting for initial osdmap 2024-05-11T12:17:00.836 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:17:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4[45670]: 2024-05-11T12:17:00.516+0000 7f39d168c700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:17:01.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:00 smithi050 ceph-mon[34072]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:17:01.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:00 smithi050 ceph-mon[34072]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:17:01.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:00 smithi050 ceph-mon[34072]: osdmap e31: 5 total, 4 up, 5 in 2024-05-11T12:17:01.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:01.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:01.126 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:00 smithi050 ceph-mon[34072]: from='osd.4 ' entity='osd.4' 2024-05-11T12:17:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:00 smithi047 ceph-mon[30375]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:17:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:00 smithi047 ceph-mon[30375]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:17:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:00 smithi047 ceph-mon[30375]: osdmap e31: 5 total, 4 up, 5 in 2024-05-11T12:17:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:00 smithi047 ceph-mon[30375]: from='osd.4 ' entity='osd.4' 2024-05-11T12:17:01.317 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:00 smithi064 ceph-mon[34537]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:17:01.317 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:00 smithi064 ceph-mon[34537]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:17:01.317 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:00 smithi064 ceph-mon[34537]: osdmap e31: 5 total, 4 up, 5 in 2024-05-11T12:17:01.317 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:01.317 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:01.317 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:00 smithi064 ceph-mon[34537]: from='osd.4 ' entity='osd.4' 2024-05-11T12:17:01.912 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-11T12:17:01.912 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-11T12:17:01.912 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-11T12:17:01.912 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-11T12:17:01.912 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113134 s, 927 MB/s 2024-05-11T12:17:01.912 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-11T12:17:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:01 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:17:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:01 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:17:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:01 smithi047 ceph-mon[30375]: osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] boot 2024-05-11T12:17:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:01 smithi047 ceph-mon[30375]: osdmap e32: 5 total, 5 up, 5 in 2024-05-11T12:17:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:01 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:17:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:01 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:17:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:01 smithi064 ceph-mon[34537]: osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] boot 2024-05-11T12:17:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:01 smithi064 ceph-mon[34537]: osdmap e32: 5 total, 5 up, 5 in 2024-05-11T12:17:02.358 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:02.372 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:01 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:17:02.372 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:01 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:17:02.372 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:02.372 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:01 smithi050 ceph-mon[34072]: osd.4 [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] boot 2024-05-11T12:17:02.373 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:01 smithi050 ceph-mon[34072]: osdmap e32: 5 total, 5 up, 5 in 2024-05-11T12:17:02.373 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-11T12:17:02.669 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_3 2024-05-11T12:17:02.933 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: osdmap e33: 5 total, 5 up, 5 in 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:03.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:03.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:02 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: osdmap e33: 5 total, 5 up, 5 in 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:03.314 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:02 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: osdmap e33: 5 total, 5 up, 5 in 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:02 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:04.065 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:03 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi050 2024-05-11T12:17:04.065 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:03 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi050 to 8546M 2024-05-11T12:17:04.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:03 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi050 2024-05-11T12:17:04.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:03 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi050 to 8546M 2024-05-11T12:17:04.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:03 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi050 2024-05-11T12:17:04.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:03 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi050 to 8546M 2024-05-11T12:17:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 -- 172.21.15.64:0/2013549230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 msgr2=0x7fcf3c104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 --2- 172.21.15.64:0/2013549230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c104c10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009990 tx=0x7fcf28009ca0 comp rx=0 tx=0).stop 2024-05-11T12:17:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 -- 172.21.15.64:0/2013549230 shutdown_connections 2024-05-11T12:17:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 --2- 172.21.15.64:0/2013549230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf3c105150 0x7fcf3c10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 --2- 172.21.15.64:0/2013549230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf3c1035b0 0x7fcf3c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:04.611 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 --2- 172.21.15.64:0/2013549230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c104c10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 -- 172.21.15.64:0/2013549230 >> 172.21.15.64:0/2013549230 conn(0x7fcf3c0feb50 msgr2=0x7fcf3c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 -- 172.21.15.64:0/2013549230 shutdown_connections 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 -- 172.21.15.64:0/2013549230 wait complete. 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 Processor -- start 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.608+0000 7fcf42c28700 1 -- start start 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf42c28700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf3c1035b0 0x7fcf3c1996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf42c28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf42c28700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf3c105150 0x7fcf3c1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf42c28700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10f020 con 0x7fcf3c1035b0 2024-05-11T12:17:04.612 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf42c28700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10eea0 con 0x7fcf3c105150 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf42c28700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3c10f1a0 con 0x7fcf3c1047b0 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:53000/0 (socket says 172.21.15.64:53000) 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 -- 172.21.15.64:0/3007494611 learned_addr learned my addr 172.21.15.64:0/3007494611 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf411c5700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf3c105150 0x7fcf3c1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf409c4700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf3c1035b0 0x7fcf3c1996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 -- 172.21.15.64:0/3007494611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf3c105150 msgr2=0x7fcf3c1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf3c105150 0x7fcf3c1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 -- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf3c1035b0 msgr2=0x7fcf3c1996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf3c1035b0 0x7fcf3c1996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 -- 172.21.15.64:0/3007494611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3c1a1350 con 0x7fcf3c1047b0 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf3bfff700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c199c30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcf280036f0 tx=0x7fcf280047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:17:04.613 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf39ffb700 1 -- 172.21.15.64:0/3007494611 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28027070 con 0x7fcf3c1047b0 2024-05-11T12:17:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.609+0000 7fcf39ffb700 1 -- 172.21.15.64:0/3007494611 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf28004000 con 0x7fcf3c1047b0 2024-05-11T12:17:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.610+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3c1a15e0 con 0x7fcf3c1047b0 2024-05-11T12:17:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.610+0000 7fcf39ffb700 1 -- 172.21.15.64:0/3007494611 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28022c60 con 0x7fcf3c1047b0 2024-05-11T12:17:04.614 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.610+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3c1a1a30 con 0x7fcf3c1047b0 2024-05-11T12:17:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.611+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3c06a240 con 0x7fcf3c1047b0 2024-05-11T12:17:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.612+0000 7fcf39ffb700 1 -- 172.21.15.64:0/3007494611 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcf2802a070 con 0x7fcf3c1047b0 2024-05-11T12:17:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.613+0000 7fcf39ffb700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcf2c0b2880 0x7fcf2c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.613+0000 7fcf39ffb700 1 -- 172.21.15.64:0/3007494611 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4101+0+0 (secure 0 0 0) 0x7fcf280e2890 con 0x7fcf3c1047b0 2024-05-11T12:17:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.613+0000 7fcf409c4700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcf2c0b2880 0x7fcf2c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:04.615 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.613+0000 7fcf409c4700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcf2c0b2880 0x7fcf2c0b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c104610 tx=0x7fcf3000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:17:04.617 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.617+0000 7fcf39ffb700 1 -- 172.21.15.64:0/3007494611 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf28031070 con 0x7fcf3c1047b0 2024-05-11T12:17:04.841 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:04.840+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcf3c063990 con 0x7fcf2c0b2880 2024-05-11T12:17:05.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:04 smithi064 ceph-mon[34537]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:05.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:04 smithi047 ceph-mon[30375]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:05.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:04 smithi050 ceph-mon[34072]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:06.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:06 smithi064 ceph-mon[34537]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:06.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:06 smithi050 ceph-mon[34072]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:06.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:06 smithi047 ceph-mon[30375]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:07.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:07 smithi047 ceph-mon[30375]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:07.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:07 smithi064 ceph-mon[34537]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:07.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:07 smithi050 ceph-mon[34072]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:08.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:08 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/1267334278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]: dispatch 2024-05-11T12:17:08.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:08 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]: dispatch 2024-05-11T12:17:08.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:08 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]': finished 2024-05-11T12:17:08.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:08 smithi047 ceph-mon[30375]: osdmap e34: 6 total, 5 up, 6 in 2024-05-11T12:17:08.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:08.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:08 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/1267334278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]: dispatch 2024-05-11T12:17:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:08 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]: dispatch 2024-05-11T12:17:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:08 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]': finished 2024-05-11T12:17:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:08 smithi064 ceph-mon[34537]: osdmap e34: 6 total, 5 up, 6 in 2024-05-11T12:17:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:08.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:08 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/1267334278' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]: dispatch 2024-05-11T12:17:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:08 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]: dispatch 2024-05-11T12:17:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:08 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "46f116be-c1a5-4e7b-b43c-3a2e79660066"}]': finished 2024-05-11T12:17:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:08 smithi050 ceph-mon[34072]: osdmap e34: 6 total, 5 up, 6 in 2024-05-11T12:17:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:09.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:09 smithi047 ceph-mon[30375]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:09.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:09 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/2936003930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:17:09.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:09 smithi064 ceph-mon[34537]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:09.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:09 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/2936003930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:17:09.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:09 smithi050 ceph-mon[34072]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:09.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:09 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/2936003930' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:17:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:11 smithi047 ceph-mon[30375]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:11.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:11 smithi064 ceph-mon[34537]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:11.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:11 smithi050 ceph-mon[34072]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:13.222 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:13 smithi064 ceph-mon[34537]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:13.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:13 smithi047 ceph-mon[30375]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:13.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:13 smithi050 ceph-mon[34072]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:15 smithi047 ceph-mon[30375]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:15 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-11T12:17:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:15 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:15.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:15 smithi064 ceph-mon[34537]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:15 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-11T12:17:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:15 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:15.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:15 smithi050 ceph-mon[34072]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:15.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:15 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-11T12:17:15.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:15 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:16.235 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:16 smithi064 ceph-mon[34537]: Deploying daemon osd.5 on smithi064 2024-05-11T12:17:16.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:16 smithi047 ceph-mon[30375]: Deploying daemon osd.5 on smithi064 2024-05-11T12:17:16.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:16 smithi050 ceph-mon[34072]: Deploying daemon osd.5 on smithi064 2024-05-11T12:17:17.265 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:17 smithi064 ceph-mon[34537]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:17.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:17 smithi050 ceph-mon[34072]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:17.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:17 smithi047 ceph-mon[30375]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:18.295 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:18 smithi064 ceph-mon[34537]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:18.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:18 smithi050 ceph-mon[34072]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:18 smithi047 ceph-mon[30375]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:20.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:20 smithi064 ceph-mon[34537]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:20.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:20 smithi050 ceph-mon[34072]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:21.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:20 smithi047 ceph-mon[30375]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:21.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:21 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:21.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:21 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:21.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:21 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:22.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:22.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:22.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:22.052 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:21 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:22.052 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:21 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:22.052 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:21 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:22.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:22 smithi050 ceph-mon[34072]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:23.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:22 smithi047 ceph-mon[30375]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:23.049 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:22 smithi064 ceph-mon[34537]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:24.513 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.512+0000 7fcf39ffb700 1 -- 172.21.15.64:0/3007494611 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcf3c063990 con 0x7fcf2c0b2880 2024-05-11T12:17:24.514 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 5 on host 'smithi064' 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcf2c0b2880 msgr2=0x7fcf2c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcf2c0b2880 0x7fcf2c0b4d40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c104610 tx=0x7fcf3000b410 comp rx=0 tx=0).stop 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 msgr2=0x7fcf3c199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c199c30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcf280036f0 tx=0x7fcf280047b0 comp rx=0 tx=0).stop 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 shutdown_connections 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcf2c0b2880 0x7fcf2c0b4d40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcf3c105150 0x7fcf3c1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcf3c1035b0 0x7fcf3c1996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 --2- 172.21.15.64:0/3007494611 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcf3c1047b0 0x7fcf3c199c30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 >> 172.21.15.64:0/3007494611 conn(0x7fcf3c0feb50 msgr2=0x7fcf3c107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:17:24.515 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 shutdown_connections 2024-05-11T12:17:24.516 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:17:24.515+0000 7fcf42c28700 1 -- 172.21.15.64:0/3007494611 wait complete. 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:24.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T12:17:24.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:24 smithi047 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T12:17:24.842 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:24.842 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T12:17:24.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:24 smithi064 ceph-mon[34537]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T12:17:24.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:24.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T12:17:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:24 smithi050 ceph-mon[34072]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-11T12:17:25.195 DEBUG:teuthology.orchestra.run.smithi064:osd.5> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.5.service 2024-05-11T12:17:25.197 INFO:tasks.cephadm:Deploying osd.6 on smithi047 with /dev/vg_nvme/lv_2... 2024-05-11T12:17:25.197 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-11T12:17:25.458 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:17:26.650 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:26 smithi064 ceph-mon[34537]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-11T12:17:26.651 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:26 smithi064 ceph-mon[34537]: osdmap e35: 6 total, 5 up, 6 in 2024-05-11T12:17:26.651 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:26 smithi064 ceph-mon[34537]: from='osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:17:26.651 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:26.651 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:26 smithi064 ceph-mon[34537]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:17:26.651 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:26 smithi064 ceph-mon[34537]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:26.651 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:17:26 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:17:26.489+0000 7f88563a4700 -1 osd.5 0 waiting for initial osdmap 2024-05-11T12:17:26.651 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:17:26 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:17:26.496+0000 7f8852197700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:17:26.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:26 smithi047 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-11T12:17:26.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:26 smithi047 ceph-mon[30375]: osdmap e35: 6 total, 5 up, 6 in 2024-05-11T12:17:26.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:26 smithi047 ceph-mon[30375]: from='osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:17:26.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:26.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:26 smithi047 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:17:26.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:26 smithi047 ceph-mon[30375]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:26.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:26 smithi050 ceph-mon[34072]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-11T12:17:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:26 smithi050 ceph-mon[34072]: osdmap e35: 6 total, 5 up, 6 in 2024-05-11T12:17:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:26 smithi050 ceph-mon[34072]: from='osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:17:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:26 smithi050 ceph-mon[34072]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:17:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:26 smithi050 ceph-mon[34072]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:27.599 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-11T12:17:27.599 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-11T12:17:27.599 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-05-11T12:17:27.600 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-05-11T12:17:27.600 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00848311 s, 1.2 GB/s 2024-05-11T12:17:27.600 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-05-11T12:17:27.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:27 smithi047 ceph-mon[30375]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:17:27.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:27 smithi047 ceph-mon[30375]: osdmap e36: 6 total, 5 up, 6 in 2024-05-11T12:17:27.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:27.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:27 smithi064 ceph-mon[34537]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:17:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:27 smithi064 ceph-mon[34537]: osdmap e36: 6 total, 5 up, 6 in 2024-05-11T12:17:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:27.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:27 smithi050 ceph-mon[34072]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:17:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:27 smithi050 ceph-mon[34072]: osdmap e36: 6 total, 5 up, 6 in 2024-05-11T12:17:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:28.352 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_2 2024-05-11T12:17:28.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:17:28.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:17:28.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:28.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] boot 2024-05-11T12:17:28.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: osdmap e37: 6 total, 6 up, 6 in 2024-05-11T12:17:28.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:28.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:28.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:28.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:28.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:28.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.617 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:17:28.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] boot 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: osdmap e37: 6 total, 6 up, 6 in 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.741 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:28.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:28.742 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: pgmap v139: 1 pgs: 1 peering; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: osd.5 [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] boot 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: osdmap e37: 6 total, 6 up, 6 in 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:28.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:29.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:29 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi064 2024-05-11T12:17:29.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:29 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-11T12:17:29.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:29 smithi047 ceph-mon[30375]: osdmap e38: 6 total, 6 up, 6 in 2024-05-11T12:17:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:29 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi064 2024-05-11T12:17:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:29 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-11T12:17:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:29 smithi064 ceph-mon[34537]: osdmap e38: 6 total, 6 up, 6 in 2024-05-11T12:17:29.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:29 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi064 2024-05-11T12:17:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:29 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi064 to 8544M 2024-05-11T12:17:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:29 smithi050 ceph-mon[34072]: osdmap e38: 6 total, 6 up, 6 in 2024-05-11T12:17:30.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 -- 172.21.15.47:0/1934549803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58109360 msgr2=0x7f6e5810b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/1934549803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58109360 0x7f6e5810b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b0a0 tx=0x7f6e5000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 -- 172.21.15.47:0/1934549803 shutdown_connections 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/1934549803 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58109360 0x7f6e5810b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/1934549803 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e58104100 0x7f6e581064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/1934549803 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e58106a30 0x7f6e58108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 -- 172.21.15.47:0/1934549803 >> 172.21.15.47:0/1934549803 conn(0x7f6e580fda70 msgr2=0x7f6e580ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 -- 172.21.15.47:0/1934549803 shutdown_connections 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.415+0000 7f6e5e80a700 1 -- 172.21.15.47:0/1934549803 wait complete. 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 Processor -- start 2024-05-11T12:17:30.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 -- start start 2024-05-11T12:17:30.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e58104100 0x7f6e58199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:30.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58106a30 0x7f6e58199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e58109360 0x7f6e581a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5810f130 con 0x7f6e58106a30 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5810efb0 con 0x7f6e58109360 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.416+0000 7f6e5e80a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5810f2b0 con 0x7f6e58104100 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58106a30 0x7f6e58199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58106a30 0x7f6e58199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46658/0 (socket says 172.21.15.47:46658) 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 -- 172.21.15.47:0/3622681252 learned_addr learned my addr 172.21.15.47:0/3622681252 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e57fff700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e58104100 0x7f6e58199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 -- 172.21.15.47:0/3622681252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e58104100 msgr2=0x7f6e58199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e58104100 0x7f6e58199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 -- 172.21.15.47:0/3622681252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e58109360 msgr2=0x7f6e581a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e58109360 0x7f6e581a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:30.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 -- 172.21.15.47:0/3622681252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e581a13e0 con 0x7f6e58106a30 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e57fff700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e58104100 0x7f6e58199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e4ffff700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58106a30 0x7f6e58199cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000b560 tx=0x7f6e4000b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e55ffb700 1 -- 172.21.15.47:0/3622681252 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40015020 con 0x7f6e58106a30 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e55ffb700 1 -- 172.21.15.47:0/3622681252 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e40011ce0 con 0x7f6e58106a30 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e581a16d0 con 0x7f6e58106a30 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e55ffb700 1 -- 172.21.15.47:0/3622681252 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40010da0 con 0x7f6e58106a30 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.417+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e581a1c90 con 0x7f6e58106a30 2024-05-11T12:17:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.418+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e5806a240 con 0x7f6e58106a30 2024-05-11T12:17:30.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.420+0000 7f6e55ffb700 1 -- 172.21.15.47:0/3622681252 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6e40018020 con 0x7f6e58106a30 2024-05-11T12:17:30.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.420+0000 7f6e55ffb700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6e440b2880 0x7f6e440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:30.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.420+0000 7f6e57fff700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6e440b2880 0x7f6e440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:30.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.420+0000 7f6e55ffb700 1 -- 172.21.15.47:0/3622681252 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 4533+0+0 (secure 0 0 0) 0x7f6e400e1130 con 0x7f6e58106a30 2024-05-11T12:17:30.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.421+0000 7f6e57fff700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6e440b2880 0x7f6e440b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007900 tx=0x7f6e48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:17:30.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.424+0000 7f6e55ffb700 1 -- 172.21.15.47:0/3622681252 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6e4001f020 con 0x7f6e58106a30 2024-05-11T12:17:30.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:30.651+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6e58063990 con 0x7f6e440b2880 2024-05-11T12:17:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:30 smithi047 ceph-mon[30375]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:30 smithi047 ceph-mon[30375]: osdmap e39: 6 total, 6 up, 6 in 2024-05-11T12:17:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:30 smithi064 ceph-mon[34537]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:30 smithi064 ceph-mon[34537]: osdmap e39: 6 total, 6 up, 6 in 2024-05-11T12:17:30.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:30 smithi050 ceph-mon[34072]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:30.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:30 smithi050 ceph-mon[34072]: osdmap e39: 6 total, 6 up, 6 in 2024-05-11T12:17:31.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:31 smithi047 ceph-mon[30375]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:31.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:31 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:31.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:31 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:31.616 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:31 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:31.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:31 smithi064 ceph-mon[34537]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:31.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:31 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:31.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:31 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:31.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:31 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:31.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:31 smithi050 ceph-mon[34072]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:31 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:31 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:31 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:32.635 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:32 smithi047 ceph-mon[30375]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:32.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:32 smithi064 ceph-mon[34537]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:32.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:32 smithi050 ceph-mon[34072]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:33.816 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:33 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3132579683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f2ec271-0e78-490a-86f4-c7058de78f6b"}]: dispatch 2024-05-11T12:17:33.817 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:33 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3132579683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f2ec271-0e78-490a-86f4-c7058de78f6b"}]': finished 2024-05-11T12:17:33.817 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:33 smithi064 ceph-mon[34537]: osdmap e40: 7 total, 6 up, 7 in 2024-05-11T12:17:33.817 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:33 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:33.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:33 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3132579683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f2ec271-0e78-490a-86f4-c7058de78f6b"}]: dispatch 2024-05-11T12:17:33.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:33 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3132579683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f2ec271-0e78-490a-86f4-c7058de78f6b"}]': finished 2024-05-11T12:17:33.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:33 smithi047 ceph-mon[30375]: osdmap e40: 7 total, 6 up, 7 in 2024-05-11T12:17:33.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:33 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:33 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3132579683' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f2ec271-0e78-490a-86f4-c7058de78f6b"}]: dispatch 2024-05-11T12:17:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:33 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3132579683' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f2ec271-0e78-490a-86f4-c7058de78f6b"}]': finished 2024-05-11T12:17:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:33 smithi050 ceph-mon[34072]: osdmap e40: 7 total, 6 up, 7 in 2024-05-11T12:17:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:33 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:34.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:34 smithi064 ceph-mon[34537]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T12:17:34.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:34 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3646874847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:17:34.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:34 smithi050 ceph-mon[34072]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T12:17:34.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:34 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3646874847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:17:35.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:34 smithi047 ceph-mon[30375]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-11T12:17:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:34 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3646874847' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:17:36.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:36 smithi064 ceph-mon[34537]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-11T12:17:36.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:36 smithi050 ceph-mon[34072]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-11T12:17:37.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:36 smithi047 ceph-mon[30375]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-05-11T12:17:38.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:38 smithi047 ceph-mon[30375]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:17:38.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:38 smithi064 ceph-mon[34537]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:17:38.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:38 smithi050 ceph-mon[34072]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:17:40.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:40 smithi064 ceph-mon[34537]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:17:40.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:40 smithi050 ceph-mon[34072]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:17:40.995 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:40 smithi047 ceph-mon[30375]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:17:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-11T12:17:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:41.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-11T12:17:41.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:41.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-11T12:17:41.992 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:42 smithi047 ceph-mon[30375]: Deploying daemon osd.6 on smithi047 2024-05-11T12:17:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:42 smithi047 ceph-mon[30375]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:17:42.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:42 smithi064 ceph-mon[34537]: Deploying daemon osd.6 on smithi047 2024-05-11T12:17:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:42 smithi064 ceph-mon[34537]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:17:42.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:42 smithi050 ceph-mon[34072]: Deploying daemon osd.6 on smithi047 2024-05-11T12:17:42.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:42 smithi050 ceph-mon[34072]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:17:44.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:44 smithi047 ceph-mon[30375]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:44 smithi064 ceph-mon[34537]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:44.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:44 smithi050 ceph-mon[34072]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:46 smithi064 ceph-mon[34537]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:46.912 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:46 smithi047 ceph-mon[30375]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:46 smithi050 ceph-mon[34072]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:48.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:48.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:48.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:48.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:48 smithi064 ceph-mon[34537]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:48.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:48.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:48.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:48.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:48 smithi050 ceph-mon[34072]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:48 smithi047 ceph-mon[30375]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:50.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:50 smithi064 ceph-mon[34537]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:50.907 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:50 smithi047 ceph-mon[30375]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:50.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:50 smithi050 ceph-mon[34072]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:50.982 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 6 on host 'smithi047' 2024-05-11T12:17:50.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.982+0000 7f6e55ffb700 1 -- 172.21.15.47:0/3622681252 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6e58063990 con 0x7f6e440b2880 2024-05-11T12:17:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6e440b2880 msgr2=0x7f6e440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6e440b2880 0x7f6e440b4d40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e48007900 tx=0x7f6e48008040 comp rx=0 tx=0).stop 2024-05-11T12:17:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58106a30 msgr2=0x7f6e58199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58106a30 0x7f6e58199cc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000b560 tx=0x7f6e4000b920 comp rx=0 tx=0).stop 2024-05-11T12:17:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 shutdown_connections 2024-05-11T12:17:50.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6e440b2880 0x7f6e440b4d40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6e58109360 0x7f6e581a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6e58104100 0x7f6e58199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 --2- 172.21.15.47:0/3622681252 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e58106a30 0x7f6e58199cc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 >> 172.21.15.47:0/3622681252 conn(0x7f6e580fda70 msgr2=0x7f6e581049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:17:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 shutdown_connections 2024-05-11T12:17:50.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:17:50.985+0000 7f6e5e80a700 1 -- 172.21.15.47:0/3622681252 wait complete. 2024-05-11T12:17:51.760 DEBUG:teuthology.orchestra.run.smithi047:osd.6> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.6.service 2024-05-11T12:17:51.762 INFO:tasks.cephadm:Deploying osd.7 on smithi050 with /dev/vg_nvme/lv_2... 2024-05-11T12:17:51.763 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-11T12:17:52.021 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:17:52.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:51 smithi047 ceph-mon[30375]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-11T12:17:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:51 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.143 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:51 smithi050 ceph-mon[34072]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-11T12:17:52.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.144 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:51 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:51 smithi064 ceph-mon[34537]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-11T12:17:52.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:52.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:51 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:52 smithi047 ceph-mon[30375]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:52 smithi047 ceph-mon[30375]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-11T12:17:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:52 smithi047 ceph-mon[30375]: osdmap e41: 7 total, 6 up, 7 in 2024-05-11T12:17:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:52 smithi047 ceph-mon[30375]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:17:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:53.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:52 smithi050 ceph-mon[34072]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:53.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:52 smithi050 ceph-mon[34072]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-11T12:17:53.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:52 smithi050 ceph-mon[34072]: osdmap e41: 7 total, 6 up, 7 in 2024-05-11T12:17:53.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:52 smithi050 ceph-mon[34072]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:17:53.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:53.265 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:52 smithi064 ceph-mon[34537]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:53.266 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:52 smithi064 ceph-mon[34537]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-11T12:17:53.266 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:52 smithi064 ceph-mon[34537]: osdmap e41: 7 total, 6 up, 7 in 2024-05-11T12:17:53.266 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:52 smithi064 ceph-mon[34537]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:17:53.266 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:53.729 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:17:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:17:53.396+0000 7f599744f700 -1 osd.6 0 waiting for initial osdmap 2024-05-11T12:17:53.729 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:17:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:17:53.403+0000 7f598f23a700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:17:54.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:53 smithi047 ceph-mon[30375]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:17:54.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:53 smithi047 ceph-mon[30375]: osdmap e42: 7 total, 6 up, 7 in 2024-05-11T12:17:54.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:54.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:54.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:53 smithi047 ceph-mon[30375]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' 2024-05-11T12:17:54.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:53 smithi050 ceph-mon[34072]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:17:54.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:53 smithi050 ceph-mon[34072]: osdmap e42: 7 total, 6 up, 7 in 2024-05-11T12:17:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:53 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:53 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:53 smithi050 ceph-mon[34072]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' 2024-05-11T12:17:54.240 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-11T12:17:54.240 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-11T12:17:54.240 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-11T12:17:54.240 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-11T12:17:54.240 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00837502 s, 1.3 GB/s 2024-05-11T12:17:54.240 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-11T12:17:54.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:53 smithi064 ceph-mon[34537]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:17:54.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:53 smithi064 ceph-mon[34537]: osdmap e42: 7 total, 6 up, 7 in 2024-05-11T12:17:54.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:53 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:54.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:53 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:54.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:53 smithi064 ceph-mon[34537]: from='osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314]' entity='osd.6' 2024-05-11T12:17:54.906 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-05-11T12:17:55.156 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:17:55.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:54 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:17:55.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:54 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:17:55.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:54 smithi050 ceph-mon[34072]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:55.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:55.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:54 smithi050 ceph-mon[34072]: osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] boot 2024-05-11T12:17:55.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:54 smithi050 ceph-mon[34072]: osdmap e43: 7 total, 7 up, 7 in 2024-05-11T12:17:55.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:54 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:54 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:17:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:54 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:17:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:54 smithi047 ceph-mon[30375]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:54 smithi047 ceph-mon[30375]: osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] boot 2024-05-11T12:17:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:54 smithi047 ceph-mon[30375]: osdmap e43: 7 total, 7 up, 7 in 2024-05-11T12:17:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:54 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:54 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:17:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:54 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:17:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:54 smithi064 ceph-mon[34537]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-05-11T12:17:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:54 smithi064 ceph-mon[34537]: osd.6 [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] boot 2024-05-11T12:17:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:54 smithi064 ceph-mon[34537]: osdmap e43: 7 total, 7 up, 7 in 2024-05-11T12:17:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:54 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-11T12:17:56.972 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:17:56.972 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi047 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi047 to 5697M 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:56.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:56.974 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:56.974 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:56 smithi050 ceph-mon[34072]: osdmap e44: 7 total, 7 up, 7 in 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi047 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi047 to 5697M 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:56 smithi047 ceph-mon[30375]: osdmap e44: 7 total, 7 up, 7 in 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 -- 172.21.15.50:0/60624696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 msgr2=0x7f45a810a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 --2- 172.21.15.50:0/60624696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a810a900 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f45a000b0d0 tx=0x7f45a000b3e0 comp rx=0 tx=0).stop 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 -- 172.21.15.50:0/60624696 shutdown_connections 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 --2- 172.21.15.50:0/60624696 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a810a900 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 --2- 172.21.15.50:0/60624696 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a81024b0 0x7f45a81028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 --2- 172.21.15.50:0/60624696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a81036b0 0x7f45a8103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 -- 172.21.15.50:0/60624696 >> 172.21.15.50:0/60624696 conn(0x7f45a80fda50 msgr2=0x7f45a80ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 -- 172.21.15.50:0/60624696 shutdown_connections 2024-05-11T12:17:57.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.081+0000 7f45aed53700 1 -- 172.21.15.50:0/60624696 wait complete. 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 Processor -- start 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 -- start start 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a81024b0 0x7f45a8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a81036b0 0x7f45a8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a81a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f45a810df50 con 0x7f45a81036b0 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f45a810ddd0 con 0x7f45a8104050 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45aed53700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f45a810e0d0 con 0x7f45a81024b0 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45ad2f0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a81a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45ad2f0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a81a0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:55320/0 (socket says 172.21.15.50:55320) 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45ad2f0700 1 -- 172.21.15.50:0/3050337347 learned_addr learned my addr 172.21.15.50:0/3050337347 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45acaef700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a81024b0 0x7f45a8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45a7fff700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a81036b0 0x7f45a8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:57.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45ad2f0700 1 -- 172.21.15.50:0/3050337347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a81024b0 msgr2=0x7f45a8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.082+0000 7f45ad2f0700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a81024b0 0x7f45a8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45ad2f0700 1 -- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a81036b0 msgr2=0x7f45a8199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45ad2f0700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a81036b0 0x7f45a8199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45ad2f0700 1 -- 172.21.15.50:0/3050337347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a81a12b0 con 0x7f45a8104050 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45a7fff700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a81036b0 0x7f45a8199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45ad2f0700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a81a0c10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f45a000d010 tx=0x7f45a0007ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45a5ffb700 1 -- 172.21.15.50:0/3050337347 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a0021070 con 0x7f45a8104050 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45a5ffb700 1 -- 172.21.15.50:0/3050337347 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45a001db00 con 0x7f45a8104050 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a81a1540 con 0x7f45a8104050 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45a5ffb700 1 -- 172.21.15.50:0/3050337347 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a001bc90 con 0x7f45a8104050 2024-05-11T12:17:57.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.083+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81a19f0 con 0x7f45a8104050 2024-05-11T12:17:57.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.084+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a806a240 con 0x7f45a8104050 2024-05-11T12:17:57.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.086+0000 7f45a5ffb700 1 -- 172.21.15.50:0/3050337347 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f45a0024070 con 0x7f45a8104050 2024-05-11T12:17:57.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.086+0000 7f45a5ffb700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f45900b2900 0x7f45900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:17:57.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.086+0000 7f45a5ffb700 1 -- 172.21.15.50:0/3050337347 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4965+0+0 (secure 0 0 0) 0x7f45a00e2c20 con 0x7f45a8104050 2024-05-11T12:17:57.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.086+0000 7f45acaef700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f45900b2900 0x7f45900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:17:57.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.087+0000 7f45acaef700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f45900b2900 0x7f45900b4dc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4598005950 tx=0x7f45980058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:17:57.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.090+0000 7f45a5ffb700 1 -- 172.21.15.50:0/3050337347 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45a00ab6e0 con 0x7f45a8104050 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi047 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi047 to 5697M 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:57.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:17:57.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:57.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:17:57.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:17:57.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:56 smithi064 ceph-mon[34537]: osdmap e44: 7 total, 7 up, 7 in 2024-05-11T12:17:57.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:17:57.313+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f45a81a1d10 con 0x7f45900b2900 2024-05-11T12:17:58.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:58.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:58.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:57 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:57 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:57 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:58.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:57 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:17:58.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:57 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:17:58.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:57 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:17:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:58 smithi047 ceph-mon[30375]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:17:58 smithi047 ceph-mon[30375]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:17:59.105 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:58 smithi050 ceph-mon[34072]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:59.105 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:17:58 smithi050 ceph-mon[34072]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:17:59.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:58 smithi064 ceph-mon[34537]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:17:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:17:58 smithi064 ceph-mon[34537]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:00.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:00 smithi050 ceph-mon[34072]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:00 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/1546484113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]: dispatch 2024-05-11T12:18:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:00 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]: dispatch 2024-05-11T12:18:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:00 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]': finished 2024-05-11T12:18:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:00 smithi050 ceph-mon[34072]: osdmap e45: 8 total, 7 up, 8 in 2024-05-11T12:18:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:00 smithi047 ceph-mon[30375]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:00 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/1546484113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]: dispatch 2024-05-11T12:18:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:00 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]: dispatch 2024-05-11T12:18:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:00 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]': finished 2024-05-11T12:18:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:00 smithi047 ceph-mon[30375]: osdmap e45: 8 total, 7 up, 8 in 2024-05-11T12:18:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:00 smithi064 ceph-mon[34537]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:00 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/1546484113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]: dispatch 2024-05-11T12:18:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:00 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]: dispatch 2024-05-11T12:18:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:00 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b473c499-1ea3-4f0c-b534-07b725db8c14"}]': finished 2024-05-11T12:18:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:00 smithi064 ceph-mon[34537]: osdmap e45: 8 total, 7 up, 8 in 2024-05-11T12:18:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:02.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:01 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/586981145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:02.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:01 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/586981145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:02.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:01 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/586981145' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:03.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:02 smithi047 ceph-mon[30375]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:03.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:02 smithi064 ceph-mon[34537]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:03.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:02 smithi050 ceph-mon[34072]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:04.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:04 smithi064 ceph-mon[34537]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:05.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:04 smithi047 ceph-mon[30375]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:05.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:04 smithi050 ceph-mon[34072]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:06 smithi047 ceph-mon[30375]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:07.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:06 smithi064 ceph-mon[34537]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:06 smithi050 ceph-mon[34072]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:08.868 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:08 smithi050 ceph-mon[34072]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:08.868 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-11T12:18:08.868 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:08.868 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:08 smithi050 ceph-mon[34072]: Deploying daemon osd.7 on smithi050 2024-05-11T12:18:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:08 smithi047 ceph-mon[30375]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-11T12:18:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:08 smithi047 ceph-mon[30375]: Deploying daemon osd.7 on smithi050 2024-05-11T12:18:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:08 smithi064 ceph-mon[34537]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-11T12:18:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:08 smithi064 ceph-mon[34537]: Deploying daemon osd.7 on smithi050 2024-05-11T12:18:10.913 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:10 smithi050 ceph-mon[34072]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:11.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:10 smithi047 ceph-mon[30375]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:11.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:10 smithi064 ceph-mon[34537]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:13.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:12 smithi047 ceph-mon[30375]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:13.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:12 smithi064 ceph-mon[34537]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:13.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:12 smithi050 ceph-mon[34072]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:14.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:14 smithi050 ceph-mon[34072]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:14.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:14.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:14.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:14 smithi047 ceph-mon[30375]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:15.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:15.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:14 smithi064 ceph-mon[34537]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:15.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:15.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:15.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:16.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:16 smithi050 ceph-mon[34072]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:17.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:16 smithi047 ceph-mon[30375]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:17.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:16 smithi064 ceph-mon[34537]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:18.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.037+0000 7f45a5ffb700 1 -- 172.21.15.50:0/3050337347 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f45a81a1d10 con 0x7f45900b2900 2024-05-11T12:18:18.037 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 7 on host 'smithi050' 2024-05-11T12:18:18.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f45900b2900 msgr2=0x7f45900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:18.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f45900b2900 0x7f45900b4dc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4598005950 tx=0x7f45980058e0 comp rx=0 tx=0).stop 2024-05-11T12:18:18.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 msgr2=0x7f45a81a0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a81a0c10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f45a000d010 tx=0x7f45a0007ad0 comp rx=0 tx=0).stop 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 shutdown_connections 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f45900b2900 0x7f45900b4dc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f45a8104050 0x7f45a81a0c10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f45a81024b0 0x7f45a8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 --2- 172.21.15.50:0/3050337347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f45a81036b0 0x7f45a8199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.039+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 >> 172.21.15.50:0/3050337347 conn(0x7f45a80fda50 msgr2=0x7f45a80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.040+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 shutdown_connections 2024-05-11T12:18:18.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:18:18.040+0000 7f45aed53700 1 -- 172.21.15.50:0/3050337347 wait complete. 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T12:18:18.352 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.353 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.353 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:18 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:18 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:18.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:18 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:18.672 DEBUG:teuthology.orchestra.run.smithi050:osd.7> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.7.service 2024-05-11T12:18:18.675 INFO:tasks.cephadm:Deploying osd.8 on smithi064 with /dev/vg_nvme/lv_2... 2024-05-11T12:18:18.675 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-11T12:18:18.933 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:18:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:19 smithi064 ceph-mon[34537]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-11T12:18:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:19 smithi064 ceph-mon[34537]: osdmap e46: 8 total, 7 up, 8 in 2024-05-11T12:18:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:19 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:19 smithi064 ceph-mon[34537]: from='osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:18:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:19 smithi064 ceph-mon[34537]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:18:19.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:19 smithi050 ceph-mon[34072]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-11T12:18:19.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:19 smithi050 ceph-mon[34072]: osdmap e46: 8 total, 7 up, 8 in 2024-05-11T12:18:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:19 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:19 smithi050 ceph-mon[34072]: from='osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:18:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:19 smithi050 ceph-mon[34072]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:18:19.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:19 smithi047 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-11T12:18:19.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:19 smithi047 ceph-mon[30375]: osdmap e46: 8 total, 7 up, 8 in 2024-05-11T12:18:19.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:19 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:19.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:19 smithi047 ceph-mon[30375]: from='osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:18:19.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:19 smithi047 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:18:19.918 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:18:19 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:18:19.604+0000 7f06ca695700 -1 osd.7 0 waiting for initial osdmap 2024-05-11T12:18:19.918 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:18:19 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:18:19.613+0000 7f06c2480700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:18:20.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:20 smithi064 ceph-mon[34537]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:18:20.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:20 smithi064 ceph-mon[34537]: osdmap e47: 8 total, 7 up, 8 in 2024-05-11T12:18:20.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:20.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:20.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:20 smithi064 ceph-mon[34537]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:20.361 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:20 smithi064 ceph-mon[34537]: from='osd.7 ' entity='osd.7' 2024-05-11T12:18:20.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:20 smithi050 ceph-mon[34072]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:18:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:20 smithi050 ceph-mon[34072]: osdmap e47: 8 total, 7 up, 8 in 2024-05-11T12:18:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:20 smithi050 ceph-mon[34072]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:20 smithi050 ceph-mon[34072]: from='osd.7 ' entity='osd.7' 2024-05-11T12:18:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:20 smithi047 ceph-mon[30375]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:18:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:20 smithi047 ceph-mon[30375]: osdmap e47: 8 total, 7 up, 8 in 2024-05-11T12:18:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:20 smithi047 ceph-mon[30375]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-11T12:18:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:20 smithi047 ceph-mon[30375]: from='osd.7 ' entity='osd.7' 2024-05-11T12:18:20.970 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-11T12:18:20.970 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-11T12:18:20.970 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-11T12:18:20.970 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-11T12:18:20.970 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113273 s, 926 MB/s 2024-05-11T12:18:20.970 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-11T12:18:21.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:21 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:18:21.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:21 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:18:21.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:21 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:21.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:21 smithi064 ceph-mon[34537]: osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] boot 2024-05-11T12:18:21.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:21 smithi064 ceph-mon[34537]: osdmap e48: 8 total, 8 up, 8 in 2024-05-11T12:18:21.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:21 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:21.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:21 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:18:21.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:21 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:18:21.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:21 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:21.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:21 smithi050 ceph-mon[34072]: osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] boot 2024-05-11T12:18:21.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:21 smithi050 ceph-mon[34072]: osdmap e48: 8 total, 8 up, 8 in 2024-05-11T12:18:21.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:21 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:21.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:21 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:18:21.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:21 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:18:21.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:21.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:21 smithi047 ceph-mon[30375]: osd.7 [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] boot 2024-05-11T12:18:21.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:21 smithi047 ceph-mon[30375]: osdmap e48: 8 total, 8 up, 8 in 2024-05-11T12:18:21.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:21 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-11T12:18:21.655 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_2 2024-05-11T12:18:21.910 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:18:22.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:22 smithi064 ceph-mon[34537]: osdmap e49: 8 total, 8 up, 8 in 2024-05-11T12:18:22.293 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:22 smithi064 ceph-mon[34537]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:22.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:22 smithi050 ceph-mon[34072]: osdmap e49: 8 total, 8 up, 8 in 2024-05-11T12:18:22.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:22 smithi050 ceph-mon[34072]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:22 smithi047 ceph-mon[30375]: osdmap e49: 8 total, 8 up, 8 in 2024-05-11T12:18:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:22 smithi047 ceph-mon[30375]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:23.356 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: osdmap e50: 8 total, 8 up, 8 in 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi050 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi050 to 5697M 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:23.357 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:23 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: osdmap e50: 8 total, 8 up, 8 in 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi050 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi050 to 5697M 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:23.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:23 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.580 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 -- 172.21.15.64:0/2863770832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4104040 msgr2=0x7f92c4106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2863770832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4104040 0x7f92c4106430 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f92b400b0a0 tx=0x7f92b400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 -- 172.21.15.64:0/2863770832 shutdown_connections 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2863770832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41092a0 0x7f92c410b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2863770832 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4104040 0x7f92c4106430 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2863770832 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c4106970 0x7f92c4108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 -- 172.21.15.64:0/2863770832 >> 172.21.15.64:0/2863770832 conn(0x7f92c40fda50 msgr2=0x7f92c40ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 -- 172.21.15.64:0/2863770832 shutdown_connections 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.580+0000 7f92c8f16700 1 -- 172.21.15.64:0/2863770832 wait complete. 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 Processor -- start 2024-05-11T12:18:23.581 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 -- start start 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c4104040 0x7f92c419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4106970 0x7f92c419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41092a0 0x7f92c41a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92c410f100 con 0x7f92c41092a0 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f92c410ef80 con 0x7f92c4104040 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c8f16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f92c410f280 con 0x7f92c4106970 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4106970 0x7f92c419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4106970 0x7f92c419bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:36452/0 (socket says 172.21.15.64:36452) 2024-05-11T12:18:23.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 -- 172.21.15.64:0/2445628479 learned_addr learned my addr 172.21.15.64:0/2445628479 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c4104040 0x7f92c419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c2d9d700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41092a0 0x7f92c41a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 -- 172.21.15.64:0/2445628479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c4104040 msgr2=0x7f92c419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c4104040 0x7f92c419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 -- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41092a0 msgr2=0x7f92c41a2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41092a0 0x7f92c41a2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.581+0000 7f92c1d9b700 1 -- 172.21.15.64:0/2445628479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c41a3680 con 0x7f92c4106970 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.582+0000 7f92c2d9d700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41092a0 0x7f92c41a2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.582+0000 7f92c1d9b700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4106970 0x7f92c419bf60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00d980 tx=0x7f92ac00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.582+0000 7f92bb7fe700 1 -- 172.21.15.64:0/2445628479 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac016070 con 0x7f92c4106970 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.582+0000 7f92bb7fe700 1 -- 172.21.15.64:0/2445628479 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92ac00de70 con 0x7f92c4106970 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.582+0000 7f92bb7fe700 1 -- 172.21.15.64:0/2445628479 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00f3c0 con 0x7f92c4106970 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.582+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c41a3970 con 0x7f92c4106970 2024-05-11T12:18:23.584 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.582+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c41a3f30 con 0x7f92c4106970 2024-05-11T12:18:23.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.583+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c406a240 con 0x7f92c4106970 2024-05-11T12:18:23.585 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.585+0000 7f92bb7fe700 1 -- 172.21.15.64:0/2445628479 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f92ac019070 con 0x7f92c4106970 2024-05-11T12:18:23.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.586+0000 7f92bb7fe700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f92b00b2880 0x7f92b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:23.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.586+0000 7f92bb7fe700 1 -- 172.21.15.64:0/2445628479 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5397+0+0 (secure 0 0 0) 0x7f92ac0e11d0 con 0x7f92c4106970 2024-05-11T12:18:23.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.586+0000 7f92c259c700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f92b00b2880 0x7f92b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:23.586 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.587+0000 7f92c259c700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f92b00b2880 0x7f92b00b4d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009fd0 tx=0x7f92b4006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:18:23.589 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.590+0000 7f92bb7fe700 1 -- 172.21.15.64:0/2445628479 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f92ac0a9a50 con 0x7f92c4106970 2024-05-11T12:18:23.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: osdmap e50: 8 total, 8 up, 8 in 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi050 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi050 to 5697M 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:23 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:23.814 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:23.814+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f92c4063990 con 0x7f92b00b2880 2024-05-11T12:18:24.464 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:24 smithi064 ceph-mon[34537]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:24.464 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:24 smithi064 ceph-mon[34537]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:18:24.464 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:18:24.464 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:18:24.464 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:24 smithi047 ceph-mon[30375]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:24 smithi047 ceph-mon[30375]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:18:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:18:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:18:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:24.600 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:24 smithi050 ceph-mon[34072]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:24.600 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:24 smithi050 ceph-mon[34072]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:18:24.600 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:18:24.600 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:18:24.600 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:26.861 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:26 smithi050 ceph-mon[34072]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:26.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:26 smithi064 ceph-mon[34537]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:27.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:26 smithi047 ceph-mon[30375]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:27 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/2512247757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]: dispatch 2024-05-11T12:18:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:27 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]: dispatch 2024-05-11T12:18:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:27 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]': finished 2024-05-11T12:18:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:27 smithi064 ceph-mon[34537]: osdmap e51: 9 total, 8 up, 9 in 2024-05-11T12:18:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:27 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/2366914063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:27 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/2512247757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]: dispatch 2024-05-11T12:18:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:27 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]: dispatch 2024-05-11T12:18:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:27 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]': finished 2024-05-11T12:18:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:27 smithi050 ceph-mon[34072]: osdmap e51: 9 total, 8 up, 9 in 2024-05-11T12:18:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:27.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:27 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/2366914063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:27 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/2512247757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]: dispatch 2024-05-11T12:18:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:27 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]: dispatch 2024-05-11T12:18:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:27 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e67c89b1-f9de-46be-b19d-c60f72ce48f0"}]': finished 2024-05-11T12:18:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:27 smithi047 ceph-mon[30375]: osdmap e51: 9 total, 8 up, 9 in 2024-05-11T12:18:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:28.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:27 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/2366914063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:28 smithi050 ceph-mon[34072]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-11T12:18:29.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:28 smithi047 ceph-mon[30375]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-11T12:18:29.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:28 smithi064 ceph-mon[34537]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-05-11T12:18:30.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:30 smithi050 ceph-mon[34072]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:18:31.005 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:30 smithi064 ceph-mon[34537]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:18:31.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:30 smithi047 ceph-mon[30375]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:18:32.703 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:32 smithi064 ceph-mon[34537]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T12:18:32.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:32 smithi050 ceph-mon[34072]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T12:18:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:32 smithi047 ceph-mon[30375]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T12:18:34.915 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:34 smithi064 ceph-mon[34537]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:18:34.915 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-11T12:18:34.915 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:34.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:34 smithi050 ceph-mon[34072]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:18:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-11T12:18:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:34 smithi047 ceph-mon[30375]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:18:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-11T12:18:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:35.768 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:35 smithi064 ceph-mon[34537]: Deploying daemon osd.8 on smithi064 2024-05-11T12:18:35.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:35 smithi050 ceph-mon[34072]: Deploying daemon osd.8 on smithi064 2024-05-11T12:18:36.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:35 smithi047 ceph-mon[30375]: Deploying daemon osd.8 on smithi064 2024-05-11T12:18:36.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:36 smithi064 ceph-mon[34537]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:18:36.935 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:36 smithi050 ceph-mon[34072]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:18:37.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:36 smithi047 ceph-mon[30375]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:18:38.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:38 smithi064 ceph-mon[34537]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:38.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:38 smithi050 ceph-mon[34072]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:39.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:38 smithi047 ceph-mon[30375]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:40 smithi064 ceph-mon[34537]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:41.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:40 smithi050 ceph-mon[34072]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:41.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:40 smithi047 ceph-mon[30375]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:41.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:41.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:41.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:42 smithi064 ceph-mon[34537]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:43.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:42 smithi050 ceph-mon[34072]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:43.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:42 smithi047 ceph-mon[30375]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:43.285 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 8 on host 'smithi064' 2024-05-11T12:18:43.285 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.283+0000 7f92bb7fe700 1 -- 172.21.15.64:0/2445628479 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f92c4063990 con 0x7f92b00b2880 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f92b00b2880 msgr2=0x7f92b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f92b00b2880 0x7f92b00b4d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f92b4009fd0 tx=0x7f92b4006210 comp rx=0 tx=0).stop 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4106970 msgr2=0x7f92c419bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4106970 0x7f92c419bf60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00d980 tx=0x7f92ac00dc90 comp rx=0 tx=0).stop 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 shutdown_connections 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c41092a0 0x7f92c41a2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f92b00b2880 0x7f92b00b4d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f92c4104040 0x7f92c419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:43.287 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 --2- 172.21.15.64:0/2445628479 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f92c4106970 0x7f92c419bf60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:43.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 >> 172.21.15.64:0/2445628479 conn(0x7f92c40fda50 msgr2=0x7f92c4103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:18:43.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 shutdown_connections 2024-05-11T12:18:43.288 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:18:43.286+0000 7f92c8f16700 1 -- 172.21.15.64:0/2445628479 wait complete. 2024-05-11T12:18:44.009 DEBUG:teuthology.orchestra.run.smithi064:osd.8> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.8.service 2024-05-11T12:18:44.011 INFO:tasks.cephadm:Deploying osd.9 on smithi047 with /dev/vg_nvme/lv_1... 2024-05-11T12:18:44.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-11T12:18:44.275 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:18:44.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:44 smithi047 ceph-mon[30375]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:44.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.399 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.400 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:44 smithi047 ceph-mon[30375]: from='osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T12:18:44.400 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:44 smithi047 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T12:18:44.491 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.492 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.492 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:44 smithi064 ceph-mon[34537]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:44.492 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.492 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.492 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:44 smithi064 ceph-mon[34537]: from='osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T12:18:44.492 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:44 smithi064 ceph-mon[34537]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T12:18:44.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:44 smithi050 ceph-mon[34072]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:44.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:44.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:44 smithi050 ceph-mon[34072]: from='osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T12:18:44.700 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:44 smithi050 ceph-mon[34072]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-11T12:18:45.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:45 smithi047 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-11T12:18:45.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:45 smithi047 ceph-mon[30375]: osdmap e52: 9 total, 8 up, 9 in 2024-05-11T12:18:45.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:45 smithi047 ceph-mon[30375]: from='osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:18:45.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:45 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:45.826 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:45 smithi047 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:18:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:45 smithi064 ceph-mon[34537]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-11T12:18:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:45 smithi064 ceph-mon[34537]: osdmap e52: 9 total, 8 up, 9 in 2024-05-11T12:18:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:45 smithi064 ceph-mon[34537]: from='osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:18:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:45 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:45 smithi064 ceph-mon[34537]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:18:45.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:45 smithi050 ceph-mon[34072]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-11T12:18:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:45 smithi050 ceph-mon[34072]: osdmap e52: 9 total, 8 up, 9 in 2024-05-11T12:18:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:45 smithi050 ceph-mon[34072]: from='osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:18:45.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:45 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:45.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:45 smithi050 ceph-mon[34072]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:18:46.406 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:18:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:18:46.122+0000 7f7347551700 -1 osd.8 0 waiting for initial osdmap 2024-05-11T12:18:46.406 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:18:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:18:46.131+0000 7f733f33c700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:18:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-11T12:18:46.616 INFO:teuthology.orchestra.run.smithi047.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-11T12:18:46.616 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 10+0 records in 2024-05-11T12:18:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:10+0 records out 2024-05-11T12:18:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00812503 s, 1.3 GB/s 2024-05-11T12:18:46.617 INFO:teuthology.orchestra.run.smithi047.stderr: stderr: 2024-05-11T12:18:46.617 INFO:teuthology.orchestra.run.smithi047.stderr:--> Zapping successful for: 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: osdmap e53: 9 total, 8 up, 9 in 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: from='osd.8 ' entity='osd.8' 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] boot 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: osdmap e54: 9 total, 9 up, 9 in 2024-05-11T12:18:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: osdmap e53: 9 total, 8 up, 9 in 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: from='osd.8 ' entity='osd.8' 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] boot 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: osdmap e54: 9 total, 9 up, 9 in 2024-05-11T12:18:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.951 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-05-11T12:18:46.952 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:18:46.952 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: osdmap e53: 9 total, 8 up, 9 in 2024-05-11T12:18:46.952 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.952 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:46.952 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: from='osd.8 ' entity='osd.8' 2024-05-11T12:18:46.952 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: osd.8 [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] boot 2024-05-11T12:18:46.952 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: osdmap e54: 9 total, 9 up, 9 in 2024-05-11T12:18:46.953 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-11T12:18:47.400 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi047:vg_nvme/lv_1 2024-05-11T12:18:47.657 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:18:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:18:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:18:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: osdmap e55: 9 total, 9 up, 9 in 2024-05-11T12:18:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:47.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: osdmap e55: 9 total, 9 up, 9 in 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.919 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.920 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:47.920 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:47.920 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:18:47.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: osdmap e55: 9 total, 9 up, 9 in 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi064 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:48 smithi047 ceph-mon[30375]: osdmap e56: 9 total, 9 up, 9 in 2024-05-11T12:18:48.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi064 2024-05-11T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-11T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:48 smithi050 ceph-mon[34072]: osdmap e56: 9 total, 9 up, 9 in 2024-05-11T12:18:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi064 2024-05-11T12:18:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi064 to 5696M 2024-05-11T12:18:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:18:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:18:49.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:49.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:18:49.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:18:49.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:48 smithi064 ceph-mon[34537]: osdmap e56: 9 total, 9 up, 9 in 2024-05-11T12:18:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 -- 172.21.15.47:0/854962933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 msgr2=0x7f084c10a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 --2- 172.21.15.47:0/854962933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c10a900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f084400b0d0 tx=0x7f084400b3e0 comp rx=0 tx=0).stop 2024-05-11T12:18:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 -- 172.21.15.47:0/854962933 shutdown_connections 2024-05-11T12:18:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 --2- 172.21.15.47:0/854962933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c10a900 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 --2- 172.21.15.47:0/854962933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084c1024b0 0x7f084c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:49.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 --2- 172.21.15.47:0/854962933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f084c1036b0 0x7f084c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 -- 172.21.15.47:0/854962933 >> 172.21.15.47:0/854962933 conn(0x7f084c0fda50 msgr2=0x7f084c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:18:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 -- 172.21.15.47:0/854962933 shutdown_connections 2024-05-11T12:18:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.369+0000 7f085449a700 1 -- 172.21.15.47:0/854962933 wait complete. 2024-05-11T12:18:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 Processor -- start 2024-05-11T12:18:49.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 -- start start 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f084c1024b0 0x7f084c106440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084c1036b0 0x7f084c104a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c104fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10dfd0 con 0x7f084c104050 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10de50 con 0x7f084c1024b0 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f085449a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f084c10e150 con 0x7f084c1036b0 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c104fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c104fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53264/0 (socket says 172.21.15.47:53264) 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 -- 172.21.15.47:0/4172700242 learned_addr learned my addr 172.21.15.47:0/4172700242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852236700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f084c1024b0 0x7f084c106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:49.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0851a35700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084c1036b0 0x7f084c104a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 -- 172.21.15.47:0/4172700242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084c1036b0 msgr2=0x7f084c104a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084c1036b0 0x7f084c104a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 -- 172.21.15.47:0/4172700242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f084c1024b0 msgr2=0x7f084c106440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f084c1024b0 0x7f084c106440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 -- 172.21.15.47:0/4172700242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084c105880 con 0x7f084c104050 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.370+0000 7f0852a37700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c104fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0844009040 tx=0x7f0844004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.371+0000 7f083f7fe700 1 -- 172.21.15.47:0/4172700242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f084401f030 con 0x7f084c104050 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.371+0000 7f083f7fe700 1 -- 172.21.15.47:0/4172700242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0844003a00 con 0x7f084c104050 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.371+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084c1a3830 con 0x7f084c104050 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.371+0000 7f083f7fe700 1 -- 172.21.15.47:0/4172700242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08440047c0 con 0x7f084c104050 2024-05-11T12:18:49.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.371+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084c1a3d00 con 0x7f084c104050 2024-05-11T12:18:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.372+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f084c06a240 con 0x7f084c104050 2024-05-11T12:18:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.374+0000 7f083f7fe700 1 -- 172.21.15.47:0/4172700242 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0844003bb0 con 0x7f084c104050 2024-05-11T12:18:49.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.375+0000 7f083f7fe700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08380b2880 0x7f08380b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:18:49.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.375+0000 7f0852236700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08380b2880 0x7f08380b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:18:49.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.375+0000 7f083f7fe700 1 -- 172.21.15.47:0/4172700242 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f084400f020 con 0x7f084c104050 2024-05-11T12:18:49.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.375+0000 7f0852236700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08380b2880 0x7f08380b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f084c103510 tx=0x7f0840009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:18:49.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.378+0000 7f083f7fe700 1 -- 172.21.15.47:0/4172700242 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0844029020 con 0x7f084c104050 2024-05-11T12:18:49.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:18:49.602+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f084c0ffc00 con 0x7f08380b2880 2024-05-11T12:18:49.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:49 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:18:49.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:49 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:18:49.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:49 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:49 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:18:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:49 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:18:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:49 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:50.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:49 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:18:50.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:49 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:18:50.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:49 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:18:50.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:50 smithi050 ceph-mon[34072]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:18:50.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:50 smithi050 ceph-mon[34072]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:18:51.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:50 smithi047 ceph-mon[30375]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:18:51.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:50 smithi047 ceph-mon[30375]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:18:51.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:50 smithi064 ceph-mon[34537]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:18:51.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:50 smithi064 ceph-mon[34537]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi047:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:18:52.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:52 smithi050 ceph-mon[34072]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-11T12:18:53.013 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:52 smithi047 ceph-mon[30375]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-11T12:18:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:52 smithi064 ceph-mon[34537]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-05-11T12:18:53.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:53 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1165029292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000a3e9c-c28a-46e4-b668-b809b80e11cd"}]: dispatch 2024-05-11T12:18:53.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:53 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1165029292' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "000a3e9c-c28a-46e4-b668-b809b80e11cd"}]': finished 2024-05-11T12:18:53.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:53 smithi050 ceph-mon[34072]: osdmap e57: 10 total, 9 up, 10 in 2024-05-11T12:18:53.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:53 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:18:53.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:53 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/13748016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:54.015 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:53 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1165029292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000a3e9c-c28a-46e4-b668-b809b80e11cd"}]: dispatch 2024-05-11T12:18:54.016 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:53 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1165029292' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "000a3e9c-c28a-46e4-b668-b809b80e11cd"}]': finished 2024-05-11T12:18:54.016 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:53 smithi064 ceph-mon[34537]: osdmap e57: 10 total, 9 up, 10 in 2024-05-11T12:18:54.016 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:53 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:18:54.016 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:53 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/13748016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:53 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1165029292' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "000a3e9c-c28a-46e4-b668-b809b80e11cd"}]: dispatch 2024-05-11T12:18:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:53 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1165029292' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "000a3e9c-c28a-46e4-b668-b809b80e11cd"}]': finished 2024-05-11T12:18:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:53 smithi047 ceph-mon[30375]: osdmap e57: 10 total, 9 up, 10 in 2024-05-11T12:18:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:18:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:53 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/13748016' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:18:54.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:54 smithi050 ceph-mon[34072]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-11T12:18:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:54 smithi047 ceph-mon[30375]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-11T12:18:55.034 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:54 smithi064 ceph-mon[34537]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-11T12:18:56.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:56 smithi050 ceph-mon[34072]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:18:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:56 smithi047 ceph-mon[30375]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:18:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:56 smithi064 ceph-mon[34537]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:18:58.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:18:58 smithi047 ceph-mon[30375]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T12:18:58.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:18:58 smithi050 ceph-mon[34072]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T12:18:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:18:58 smithi064 ceph-mon[34537]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-05-11T12:19:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:00 smithi050 ceph-mon[34072]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-11T12:19:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:00.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:00 smithi047 ceph-mon[30375]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:00.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-11T12:19:00.950 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:00 smithi064 ceph-mon[34537]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-11T12:19:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:01 smithi047 ceph-mon[30375]: Deploying daemon osd.9 on smithi047 2024-05-11T12:19:01.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:01 smithi050 ceph-mon[34072]: Deploying daemon osd.9 on smithi047 2024-05-11T12:19:02.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:01 smithi064 ceph-mon[34537]: Deploying daemon osd.9 on smithi047 2024-05-11T12:19:02.920 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:02 smithi047 ceph-mon[30375]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:02.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:02 smithi050 ceph-mon[34072]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:03.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:02 smithi064 ceph-mon[34537]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:04.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:04 smithi050 ceph-mon[34072]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:04 smithi047 ceph-mon[30375]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:04 smithi064 ceph-mon[34537]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:06.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:06 smithi047 ceph-mon[30375]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:06.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:06.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:06.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:06.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:06 smithi050 ceph-mon[34072]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:06.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:06 smithi064 ceph-mon[34537]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:08 smithi050 ceph-mon[34072]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:08 smithi047 ceph-mon[30375]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:09.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:08 smithi064 ceph-mon[34537]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:10.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.016+0000 7f083f7fe700 1 -- 172.21.15.47:0/4172700242 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f084c0ffc00 con 0x7f08380b2880 2024-05-11T12:19:10.016 INFO:teuthology.orchestra.run.smithi047.stdout:Created osd(s) 9 on host 'smithi047' 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08380b2880 msgr2=0x7f08380b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08380b2880 0x7f08380b4d40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f084c103510 tx=0x7f0840009190 comp rx=0 tx=0).stop 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 msgr2=0x7f084c104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c104fd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0844009040 tx=0x7f0844004d50 comp rx=0 tx=0).stop 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 shutdown_connections 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f084c104050 0x7f084c104fd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f08380b2880 0x7f08380b4d40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f084c1024b0 0x7f084c106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:10.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 --2- 172.21.15.47:0/4172700242 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f084c1036b0 0x7f084c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 >> 172.21.15.47:0/4172700242 conn(0x7f084c0fda50 msgr2=0x7f084c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:19:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 shutdown_connections 2024-05-11T12:19:10.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:19:10.019+0000 7f085449a700 1 -- 172.21.15.47:0/4172700242 wait complete. 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:10.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:10 smithi047 ceph-mon[30375]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-11T12:19:10.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:10 smithi050 ceph-mon[34072]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:10.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:10.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:10 smithi064 ceph-mon[34537]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-11T12:19:10.660 DEBUG:teuthology.orchestra.run.smithi047:osd.9> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.9.service 2024-05-11T12:19:10.662 INFO:tasks.cephadm:Deploying osd.10 on smithi050 with /dev/vg_nvme/lv_1... 2024-05-11T12:19:10.662 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-11T12:19:10.920 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:19:12.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:11 smithi047 ceph-mon[30375]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-11T12:19:12.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:11 smithi047 ceph-mon[30375]: osdmap e58: 10 total, 9 up, 10 in 2024-05-11T12:19:12.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:11 smithi047 ceph-mon[30375]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:19:12.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:12.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:11 smithi050 ceph-mon[34072]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-11T12:19:12.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:11 smithi050 ceph-mon[34072]: osdmap e58: 10 total, 9 up, 10 in 2024-05-11T12:19:12.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:11 smithi050 ceph-mon[34072]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:19:12.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:12 smithi064 ceph-mon[34537]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-11T12:19:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:12 smithi064 ceph-mon[34537]: osdmap e58: 10 total, 9 up, 10 in 2024-05-11T12:19:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:12 smithi064 ceph-mon[34537]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]: dispatch 2024-05-11T12:19:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:12 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:12.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:19:12 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:19:12.462+0000 7f2299182700 -1 osd.9 0 waiting for initial osdmap 2024-05-11T12:19:12.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:19:12 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:19:12.477+0000 7f2290f6d700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:19:13.183 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-11T12:19:13.183 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-11T12:19:13.183 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-05-11T12:19:13.183 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-05-11T12:19:13.183 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00835149 s, 1.3 GB/s 2024-05-11T12:19:13.183 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-05-11T12:19:13.207 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:13 smithi050 ceph-mon[34072]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:13.207 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:13 smithi050 ceph-mon[34072]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:19:13.207 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:13 smithi050 ceph-mon[34072]: osdmap e59: 10 total, 9 up, 10 in 2024-05-11T12:19:13.207 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:13 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:13.207 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:13 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:13.207 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:13 smithi050 ceph-mon[34072]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' 2024-05-11T12:19:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:13 smithi047 ceph-mon[30375]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:13 smithi047 ceph-mon[30375]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:19:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:13 smithi047 ceph-mon[30375]: osdmap e59: 10 total, 9 up, 10 in 2024-05-11T12:19:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:13 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:13 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:13 smithi047 ceph-mon[30375]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' 2024-05-11T12:19:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:13 smithi064 ceph-mon[34537]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:13 smithi064 ceph-mon[34537]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi047", "root=default"]}]': finished 2024-05-11T12:19:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:13 smithi064 ceph-mon[34537]: osdmap e59: 10 total, 9 up, 10 in 2024-05-11T12:19:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:13 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:13 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:13 smithi064 ceph-mon[34537]: from='osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969]' entity='osd.9' 2024-05-11T12:19:13.838 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-05-11T12:19:14.106 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.b/config 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] boot 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: osdmap e60: 10 total, 10 up, 10 in 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:14.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:14.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:19:14.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:19:14.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] boot 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: osdmap e60: 10 total, 10 up, 10 in 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:14.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:14.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: osd.9 [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] boot 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: osdmap e60: 10 total, 10 up, 10 in 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:14.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:14.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:14.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:14.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:15 smithi047 ceph-mon[30375]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:15 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi047 2024-05-11T12:19:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:15 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi047 to 4273M 2024-05-11T12:19:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:15 smithi047 ceph-mon[30375]: osdmap e61: 10 total, 10 up, 10 in 2024-05-11T12:19:15.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:15 smithi050 ceph-mon[34072]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:15.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:15 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi047 2024-05-11T12:19:15.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:15 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi047 to 4273M 2024-05-11T12:19:15.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:15 smithi050 ceph-mon[34072]: osdmap e61: 10 total, 10 up, 10 in 2024-05-11T12:19:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:15 smithi064 ceph-mon[34537]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-05-11T12:19:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:15 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi047 2024-05-11T12:19:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:15 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi047 to 4273M 2024-05-11T12:19:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:15 smithi064 ceph-mon[34537]: osdmap e61: 10 total, 10 up, 10 in 2024-05-11T12:19:15.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 -- 172.21.15.50:0/3267960188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e41035b0 msgr2=0x7f04e41039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:15.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 --2- 172.21.15.50:0/3267960188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e41035b0 0x7f04e41039d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f04d40099c0 tx=0x7f04d4009cd0 comp rx=0 tx=0).stop 2024-05-11T12:19:15.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 -- 172.21.15.50:0/3267960188 shutdown_connections 2024-05-11T12:19:15.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 --2- 172.21.15.50:0/3267960188 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e4105150 0x7f04e410ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:15.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 --2- 172.21.15.50:0/3267960188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e41035b0 0x7f04e41039d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:15.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 --2- 172.21.15.50:0/3267960188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04e41047b0 0x7f04e4104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:15.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 -- 172.21.15.50:0/3267960188 >> 172.21.15.50:0/3267960188 conn(0x7f04e40feb50 msgr2=0x7f04e4100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:19:15.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 -- 172.21.15.50:0/3267960188 shutdown_connections 2024-05-11T12:19:15.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 -- 172.21.15.50:0/3267960188 wait complete. 2024-05-11T12:19:15.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 Processor -- start 2024-05-11T12:19:15.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.838+0000 7f04e9a34700 1 -- start start 2024-05-11T12:19:15.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e9a34700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e41035b0 0x7f04e40772f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e9a34700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04e41047b0 0x7f04e4075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e9a34700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e4105150 0x7f04e4075e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e9a34700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410f050 con 0x7f04e41047b0 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e9a34700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410eed0 con 0x7f04e4105150 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e9a34700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04e410f1d0 con 0x7f04e41035b0 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e4105150 0x7f04e4075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e2ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e41035b0 0x7f04e40772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e4105150 0x7f04e4075e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:35244/0 (socket says 172.21.15.50:35244) 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 -- 172.21.15.50:0/1614607208 learned_addr learned my addr 172.21.15.50:0/1614607208 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e27fc700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04e41047b0 0x7f04e4075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 -- 172.21.15.50:0/1614607208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e41035b0 msgr2=0x7f04e40772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e41035b0 0x7f04e40772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 -- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04e41047b0 msgr2=0x7f04e4075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:15.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04e41047b0 0x7f04e4075940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 -- 172.21.15.50:0/1614607208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e4076730 con 0x7f04e4105150 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e37fe700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e4105150 0x7f04e4075e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04dc007ee0 tx=0x7f04dc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.839+0000 7f04e8a32700 1 -- 172.21.15.50:0/1614607208 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc004bb0 con 0x7f04e4105150 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.840+0000 7f04e8a32700 1 -- 172.21.15.50:0/1614607208 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04dc004d50 con 0x7f04e4105150 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.840+0000 7f04e8a32700 1 -- 172.21.15.50:0/1614607208 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc005920 con 0x7f04e4105150 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.840+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41a3900 con 0x7f04e4105150 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.840+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e41a3dc0 con 0x7f04e4105150 2024-05-11T12:19:15.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.841+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e406a240 con 0x7f04e4105150 2024-05-11T12:19:15.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.842+0000 7f04e8a32700 1 -- 172.21.15.50:0/1614607208 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f04dc01a020 con 0x7f04e4105150 2024-05-11T12:19:15.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.843+0000 7f04e8a32700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04cc0b2880 0x7f04cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:15.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.843+0000 7f04e8a32700 1 -- 172.21.15.50:0/1614607208 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f04dc0e1100 con 0x7f04e4105150 2024-05-11T12:19:15.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.843+0000 7f04e2ffd700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04cc0b2880 0x7f04cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:15.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.843+0000 7f04e2ffd700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04cc0b2880 0x7f04cc0b4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b580 tx=0x7f04d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:19:15.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:15.847+0000 7f04e8a32700 1 -- 172.21.15.50:0/1614607208 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04dc010070 con 0x7f04e4105150 2024-05-11T12:19:16.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:16.073+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f04e4063990 con 0x7f04cc0b2880 2024-05-11T12:19:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:17 smithi047 ceph-mon[30375]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:19:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:19:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:17 smithi064 ceph-mon[34537]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:19:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:19:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:17.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:17 smithi050 ceph-mon[34072]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:19:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:19:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:18.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:18 smithi050 ceph-mon[34072]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:19:18.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:18 smithi047 ceph-mon[30375]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:19:18.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:18 smithi064 ceph-mon[34537]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:19:19.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:19 smithi047 ceph-mon[30375]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:19 smithi064 ceph-mon[34537]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:19.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:19 smithi050 ceph-mon[34072]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:20.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:20 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/3957252491' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]: dispatch 2024-05-11T12:19:20.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:20 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]: dispatch 2024-05-11T12:19:20.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:20 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]': finished 2024-05-11T12:19:20.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:20 smithi047 ceph-mon[30375]: osdmap e62: 11 total, 10 up, 11 in 2024-05-11T12:19:20.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:20.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:20 smithi047 ceph-mon[30375]: from='client.? 172.21.15.50:0/599810596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:19:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:20 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/3957252491' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]: dispatch 2024-05-11T12:19:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:20 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]: dispatch 2024-05-11T12:19:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:20 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]': finished 2024-05-11T12:19:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:20 smithi064 ceph-mon[34537]: osdmap e62: 11 total, 10 up, 11 in 2024-05-11T12:19:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:20 smithi064 ceph-mon[34537]: from='client.? 172.21.15.50:0/599810596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:19:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:20 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/3957252491' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]: dispatch 2024-05-11T12:19:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:20 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]: dispatch 2024-05-11T12:19:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:20 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5dc1c98-29f5-4afb-8c44-7c2085f9c414"}]': finished 2024-05-11T12:19:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:20 smithi050 ceph-mon[34072]: osdmap e62: 11 total, 10 up, 11 in 2024-05-11T12:19:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:20 smithi050 ceph-mon[34072]: from='client.? 172.21.15.50:0/599810596' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:19:21.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:21 smithi047 ceph-mon[30375]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:21.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:21 smithi064 ceph-mon[34537]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:21.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:21 smithi050 ceph-mon[34072]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:23.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:23 smithi050 ceph-mon[34072]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:23.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:23 smithi047 ceph-mon[30375]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:23.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:23 smithi064 ceph-mon[34537]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 302 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:25.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:25 smithi047 ceph-mon[30375]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:25.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:25 smithi064 ceph-mon[34537]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:25.428 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:25 smithi050 ceph-mon[34072]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:27.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:27 smithi050 ceph-mon[34072]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:27.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-11T12:19:27.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:27.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:27 smithi047 ceph-mon[30375]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:27.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-11T12:19:27.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:27.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:27 smithi064 ceph-mon[34537]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:27.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-11T12:19:27.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:28.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:28 smithi050 ceph-mon[34072]: Deploying daemon osd.10 on smithi050 2024-05-11T12:19:28.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:28 smithi064 ceph-mon[34537]: Deploying daemon osd.10 on smithi050 2024-05-11T12:19:28.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:28 smithi047 ceph-mon[30375]: Deploying daemon osd.10 on smithi050 2024-05-11T12:19:29.176 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:29 smithi050 ceph-mon[34072]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:29 smithi064 ceph-mon[34537]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:29.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:29 smithi047 ceph-mon[30375]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:31 smithi064 ceph-mon[34537]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:31.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:31 smithi050 ceph-mon[34072]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:31.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:31 smithi047 ceph-mon[30375]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:33.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:33 smithi050 ceph-mon[34072]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:33 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:33 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:33 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:33 smithi064 ceph-mon[34537]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:33 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:33 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:33 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:33.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:33 smithi047 ceph-mon[30375]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:33.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:33 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:33.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:33 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:33.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:33 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:35.298 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:35 smithi050 ceph-mon[34072]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:35.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:35 smithi064 ceph-mon[34537]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:35.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:35 smithi047 ceph-mon[30375]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:36.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.692+0000 7f04e8a32700 1 -- 172.21.15.50:0/1614607208 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f04e4063990 con 0x7f04cc0b2880 2024-05-11T12:19:36.692 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 10 on host 'smithi050' 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04cc0b2880 msgr2=0x7f04cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04cc0b2880 0x7f04cc0b4d40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f04d400b580 tx=0x7f04d4005fb0 comp rx=0 tx=0).stop 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e4105150 msgr2=0x7f04e4075e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e4105150 0x7f04e4075e80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04dc007ee0 tx=0x7f04dc00d3b0 comp rx=0 tx=0).stop 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 shutdown_connections 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04cc0b2880 0x7f04cc0b4d40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04e4105150 0x7f04e4075e80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04e41035b0 0x7f04e40772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 --2- 172.21.15.50:0/1614607208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04e41047b0 0x7f04e4075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:36.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 >> 172.21.15.50:0/1614607208 conn(0x7f04e40feb50 msgr2=0x7f04e4107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:19:36.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 shutdown_connections 2024-05-11T12:19:36.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-11T12:19:36.695+0000 7f04e9a34700 1 -- 172.21.15.50:0/1614607208 wait complete. 2024-05-11T12:19:37.330 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:37 smithi050 ceph-mon[34072]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:37.331 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:37 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:37.331 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:37 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:37.331 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:37 smithi050 ceph-mon[34072]: from='osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T12:19:37.331 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:37 smithi050 ceph-mon[34072]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T12:19:37.363 DEBUG:teuthology.orchestra.run.smithi050:osd.10> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.10.service 2024-05-11T12:19:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:37 smithi064 ceph-mon[34537]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:37 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:37 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:37 smithi064 ceph-mon[34537]: from='osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T12:19:37.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:37 smithi064 ceph-mon[34537]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T12:19:37.366 INFO:tasks.cephadm:Deploying osd.11 on smithi064 with /dev/vg_nvme/lv_1... 2024-05-11T12:19:37.366 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-11T12:19:37.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:37 smithi047 ceph-mon[30375]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:37.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:37 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:37.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:37 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:37.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:37 smithi047 ceph-mon[30375]: from='osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T12:19:37.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:37 smithi047 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-11T12:19:37.623 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:19:38.780 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:38.780 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:38.780 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:38.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-11T12:19:38.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: osdmap e63: 11 total, 10 up, 11 in 2024-05-11T12:19:38.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: from='osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:19:38.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:38.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:38 smithi047 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:19:38.834 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:38.834 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:38.835 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:38.835 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-11T12:19:38.835 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: osdmap e63: 11 total, 10 up, 11 in 2024-05-11T12:19:38.835 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: from='osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:19:38.835 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:38.835 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:38 smithi064 ceph-mon[34537]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:19:38.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:38.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:38.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:38.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-11T12:19:38.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: osdmap e63: 11 total, 10 up, 11 in 2024-05-11T12:19:38.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: from='osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:19:38.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:38.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:38 smithi050 ceph-mon[34072]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-05-11T12:19:39.373 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:19:39 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:19:39.121+0000 7f34c774a700 -1 osd.10 0 waiting for initial osdmap 2024-05-11T12:19:39.374 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:19:39 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:19:39.132+0000 7f34bf535700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:19:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-11T12:19:39.741 INFO:teuthology.orchestra.run.smithi064.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-11T12:19:39.741 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10+0 records in 2024-05-11T12:19:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:10+0 records out 2024-05-11T12:19:39.742 INFO:teuthology.orchestra.run.smithi064.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132419 s, 792 MB/s 2024-05-11T12:19:39.742 INFO:teuthology.orchestra.run.smithi064.stderr:--> Zapping successful for: 2024-05-11T12:19:39.892 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:39 smithi064 ceph-mon[34537]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:19:39.892 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:39 smithi064 ceph-mon[34537]: osdmap e64: 11 total, 10 up, 11 in 2024-05-11T12:19:39.893 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:39 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:39.893 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:39 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:39.893 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:39 smithi064 ceph-mon[34537]: from='osd.10 ' entity='osd.10' 2024-05-11T12:19:39.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:39 smithi050 ceph-mon[34072]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:19:39.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:39 smithi050 ceph-mon[34072]: osdmap e64: 11 total, 10 up, 11 in 2024-05-11T12:19:39.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:39 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:39.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:39 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:39.944 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:39 smithi050 ceph-mon[34072]: from='osd.10 ' entity='osd.10' 2024-05-11T12:19:40.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:39 smithi047 ceph-mon[30375]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-05-11T12:19:40.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:39 smithi047 ceph-mon[30375]: osdmap e64: 11 total, 10 up, 11 in 2024-05-11T12:19:40.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:39 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:40.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:39 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:40.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:39 smithi047 ceph-mon[30375]: from='osd.10 ' entity='osd.10' 2024-05-11T12:19:40.470 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch daemon add osd smithi064:vg_nvme/lv_1 2024-05-11T12:19:40.717 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:19:40.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:40 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:19:40.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:40 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:19:40.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:40 smithi064 ceph-mon[34537]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:40.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:40 smithi064 ceph-mon[34537]: osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] boot 2024-05-11T12:19:40.843 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:40 smithi064 ceph-mon[34537]: osdmap e65: 11 total, 11 up, 11 in 2024-05-11T12:19:40.844 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:40.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:40 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:19:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:40 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:19:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:40 smithi050 ceph-mon[34072]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:40 smithi050 ceph-mon[34072]: osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] boot 2024-05-11T12:19:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:40 smithi050 ceph-mon[34072]: osdmap e65: 11 total, 11 up, 11 in 2024-05-11T12:19:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:40 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:19:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:40 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:19:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:40 smithi047 ceph-mon[30375]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 303 MiB used, 894 GiB / 894 GiB avail 2024-05-11T12:19:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:40 smithi047 ceph-mon[30375]: osd.10 [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] boot 2024-05-11T12:19:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:40 smithi047 ceph-mon[30375]: osdmap e65: 11 total, 11 up, 11 in 2024-05-11T12:19:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-11T12:19:41.850 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: osdmap e66: 11 total, 11 up, 11 in 2024-05-11T12:19:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.852 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.852 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.852 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:41.852 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:41.852 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.852 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: osdmap e66: 11 total, 11 up, 11 in 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:41.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: osdmap e66: 11 total, 11 up, 11 in 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:19:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 -- 172.21.15.64:0/1003761847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c106990 msgr2=0x7feb0c108d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 --2- 172.21.15.64:0/1003761847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c106990 0x7feb0c108d80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb00009990 tx=0x7feb00009ca0 comp rx=0 tx=0).stop 2024-05-11T12:19:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 -- 172.21.15.64:0/1003761847 shutdown_connections 2024-05-11T12:19:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 --2- 172.21.15.64:0/1003761847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c1092c0 0x7feb0c10b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 --2- 172.21.15.64:0/1003761847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0c104060 0x7feb0c106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 --2- 172.21.15.64:0/1003761847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c106990 0x7feb0c108d80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:42.345 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 -- 172.21.15.64:0/1003761847 >> 172.21.15.64:0/1003761847 conn(0x7feb0c0fdab0 msgr2=0x7feb0c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:19:42.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 -- 172.21.15.64:0/1003761847 shutdown_connections 2024-05-11T12:19:42.346 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 -- 172.21.15.64:0/1003761847 wait complete. 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.345+0000 7feb12bd0700 1 Processor -- start 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb12bd0700 1 -- start start 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb12bd0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0c104060 0x7feb0c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb12bd0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c106990 0x7feb0c199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb12bd0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c1092c0 0x7feb0c1a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb12bd0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c10f090 con 0x7feb0c104060 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb12bd0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c10ef10 con 0x7feb0c106990 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb12bd0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7feb0c10f210 con 0x7feb0c1092c0 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c1092c0 0x7feb0c1a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c1092c0 0x7feb0c1a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:35406/0 (socket says 172.21.15.64:35406) 2024-05-11T12:19:42.348 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 -- 172.21.15.64:0/640607271 learned_addr learned my addr 172.21.15.64:0/640607271 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb0bfff700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c106990 0x7feb0c199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1096c700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0c104060 0x7feb0c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 -- 172.21.15.64:0/640607271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c106990 msgr2=0x7feb0c199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c106990 0x7feb0c199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 -- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0c104060 msgr2=0x7feb0c199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0c104060 0x7feb0c199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 -- 172.21.15.64:0/640607271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0c1a1370 con 0x7feb0c1092c0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.346+0000 7feb1116d700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c1092c0 0x7feb0c1a0cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feb0400add0 tx=0x7feb0400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.347+0000 7feb09ffb700 1 -- 172.21.15.64:0/640607271 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04017020 con 0x7feb0c1092c0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.347+0000 7feb09ffb700 1 -- 172.21.15.64:0/640607271 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb0400ae00 con 0x7feb0c1092c0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.347+0000 7feb09ffb700 1 -- 172.21.15.64:0/640607271 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04005280 con 0x7feb0c1092c0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.347+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0c1a1660 con 0x7feb0c1092c0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.347+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb0c1a1ba0 con 0x7feb0c1092c0 2024-05-11T12:19:42.349 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.348+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb0c06a240 con 0x7feb0c1092c0 2024-05-11T12:19:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.349+0000 7feb09ffb700 1 -- 172.21.15.64:0/640607271 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7feb0400c890 con 0x7feb0c1092c0 2024-05-11T12:19:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.350+0000 7feb09ffb700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feaf40b2880 0x7feaf40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:19:42.350 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.350+0000 7feb09ffb700 1 -- 172.21.15.64:0/640607271 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7feb040e1140 con 0x7feb0c1092c0 2024-05-11T12:19:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.350+0000 7feb1096c700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feaf40b2880 0x7feaf40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:19:42.351 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.350+0000 7feb1096c700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feaf40b2880 0x7feaf40b4d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feafc005950 tx=0x7feafc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:19:42.354 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.355+0000 7feb09ffb700 1 -- 172.21.15.64:0/640607271 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb04021020 con 0x7feb0c1092c0 2024-05-11T12:19:42.583 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:19:42.583+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7feb0c063a50 con 0x7feaf40b2880 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi050 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: osdmap e67: 11 total, 11 up, 11 in 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:19:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:42.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi050 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: osdmap e67: 11 total, 11 up, 11 in 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:19:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi050 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi050 to 4273M 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: osdmap e67: 11 total, 11 up, 11 in 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-11T12:19:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:43 smithi064 ceph-mon[34537]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:19:44.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:43 smithi047 ceph-mon[30375]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:19:44.212 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:43 smithi050 ceph-mon[34072]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi064:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:19:44.851 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:44 smithi064 ceph-mon[34537]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:44.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:44 smithi050 ceph-mon[34072]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:44 smithi047 ceph-mon[30375]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:45 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/2325881587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]: dispatch 2024-05-11T12:19:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:45 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]: dispatch 2024-05-11T12:19:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:45 smithi050 ceph-mon[34072]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]': finished 2024-05-11T12:19:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:45 smithi050 ceph-mon[34072]: osdmap e68: 12 total, 11 up, 12 in 2024-05-11T12:19:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:45 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:19:46.002 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:45 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/2325881587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]: dispatch 2024-05-11T12:19:46.002 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:45 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]: dispatch 2024-05-11T12:19:46.002 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:45 smithi064 ceph-mon[34537]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]': finished 2024-05-11T12:19:46.002 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:45 smithi064 ceph-mon[34537]: osdmap e68: 12 total, 11 up, 12 in 2024-05-11T12:19:46.003 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:45 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:19:46.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:45 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/2325881587' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]: dispatch 2024-05-11T12:19:46.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:45 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]: dispatch 2024-05-11T12:19:46.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:45 smithi047 ceph-mon[30375]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8acbed28-84a5-4506-9bb4-ff7e65e8701b"}]': finished 2024-05-11T12:19:46.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:45 smithi047 ceph-mon[30375]: osdmap e68: 12 total, 11 up, 12 in 2024-05-11T12:19:46.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:45 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:19:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:46 smithi050 ceph-mon[34072]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:46 smithi050 ceph-mon[34072]: from='client.? 172.21.15.64:0/4274335262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:19:47.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:46 smithi047 ceph-mon[30375]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:47.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:46 smithi047 ceph-mon[30375]: from='client.? 172.21.15.64:0/4274335262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:19:47.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:46 smithi064 ceph-mon[34537]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:19:47.173 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:46 smithi064 ceph-mon[34537]: from='client.? 172.21.15.64:0/4274335262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-11T12:19:49.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:48 smithi047 ceph-mon[30375]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-11T12:19:49.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:48 smithi064 ceph-mon[34537]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-11T12:19:49.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:48 smithi050 ceph-mon[34072]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-05-11T12:19:50.992 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:50 smithi064 ceph-mon[34537]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:19:51.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:50 smithi047 ceph-mon[30375]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:19:51.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:50 smithi050 ceph-mon[34072]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-11T12:19:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:52 smithi047 ceph-mon[30375]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-11T12:19:53.061 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:52 smithi064 ceph-mon[34537]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-11T12:19:53.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:52 smithi050 ceph-mon[34072]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-05-11T12:19:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-11T12:19:54.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:54.100 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:53 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-11T12:19:54.100 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:53 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:54.102 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:53 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-11T12:19:54.103 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:53 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:19:54.713 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:54 smithi064 ceph-mon[34537]: Deploying daemon osd.11 on smithi064 2024-05-11T12:19:54.713 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:54 smithi064 ceph-mon[34537]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:54 smithi047 ceph-mon[30375]: Deploying daemon osd.11 on smithi064 2024-05-11T12:19:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:54 smithi047 ceph-mon[30375]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:55.116 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:54 smithi050 ceph-mon[34072]: Deploying daemon osd.11 on smithi064 2024-05-11T12:19:55.116 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:54 smithi050 ceph-mon[34072]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:56.846 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:56 smithi064 ceph-mon[34537]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:56 smithi047 ceph-mon[30375]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:57.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:56 smithi050 ceph-mon[34072]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-11T12:19:59.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:19:58 smithi047 ceph-mon[30375]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-11T12:19:59.078 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:19:58 smithi064 ceph-mon[34537]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-11T12:19:59.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:19:58 smithi050 ceph-mon[34072]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-11T12:20:00.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:00.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:00.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:00.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:00 smithi064 ceph-mon[34537]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:00.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:00 smithi064 ceph-mon[34537]: overall HEALTH_OK 2024-05-11T12:20:00.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:00.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:00.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:00.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:00 smithi050 ceph-mon[34072]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:00.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:00 smithi050 ceph-mon[34072]: overall HEALTH_OK 2024-05-11T12:20:00.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:00.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:00.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:00.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:00 smithi047 ceph-mon[30375]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:00.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:00 smithi047 ceph-mon[30375]: overall HEALTH_OK 2024-05-11T12:20:02.700 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:02 smithi064 ceph-mon[34537]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:02.756 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.756+0000 7feb09ffb700 1 -- 172.21.15.64:0/640607271 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7feb0c063a50 con 0x7feaf40b2880 2024-05-11T12:20:02.757 INFO:teuthology.orchestra.run.smithi064.stdout:Created osd(s) 11 on host 'smithi064' 2024-05-11T12:20:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feaf40b2880 msgr2=0x7feaf40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feaf40b2880 0x7feaf40b4d40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7feafc005950 tx=0x7feafc00b410 comp rx=0 tx=0).stop 2024-05-11T12:20:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c1092c0 msgr2=0x7feb0c1a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c1092c0 0x7feb0c1a0cd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7feb0400add0 tx=0x7feb0400c330 comp rx=0 tx=0).stop 2024-05-11T12:20:02.759 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 shutdown_connections 2024-05-11T12:20:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7feb0c1092c0 0x7feb0c1a0cd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb0c104060 0x7feb0c199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feaf40b2880 0x7feaf40b4d40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 --2- 172.21.15.64:0/640607271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7feb0c106990 0x7feb0c199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 >> 172.21.15.64:0/640607271 conn(0x7feb0c0fdab0 msgr2=0x7feb0c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 shutdown_connections 2024-05-11T12:20:02.760 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:02.759+0000 7feb12bd0700 1 -- 172.21.15.64:0/640607271 wait complete. 2024-05-11T12:20:02.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:02 smithi050 ceph-mon[34072]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:03.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:02 smithi047 ceph-mon[30375]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:03.773 DEBUG:teuthology.orchestra.run.smithi064:osd.11> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.11.service 2024-05-11T12:20:03.775 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-11T12:20:03.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-11T12:20:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:03 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:03 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:03 smithi047 ceph-mon[30375]: from='osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T12:20:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:03 smithi047 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T12:20:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:04.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:03 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:04.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:03 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:04.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:03 smithi064 ceph-mon[34537]: from='osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T12:20:04.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:03 smithi064 ceph-mon[34537]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T12:20:04.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:03 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:03 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:03 smithi050 ceph-mon[34072]: from='osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T12:20:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:03 smithi050 ceph-mon[34072]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-11T12:20:04.971 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:04.972 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-11T12:20:04.972 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: osdmap e69: 12 total, 11 up, 12 in 2024-05-11T12:20:04.972 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:04.972 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: from='osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:20:04.972 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:20:04.972 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:04.972 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: osdmap e69: 12 total, 11 up, 12 in 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: from='osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:05.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:05.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-11T12:20:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: osdmap e69: 12 total, 11 up, 12 in 2024-05-11T12:20:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: from='osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:20:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]: dispatch 2024-05-11T12:20:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:05.364 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:20:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:20:05.201+0000 7fbdcd87c700 -1 osd.11 0 waiting for initial osdmap 2024-05-11T12:20:05.365 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:20:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:20:05.214+0000 7fbdc5667700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-11T12:20:05.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 -- 172.21.15.47:0/795271164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1047a0 msgr2=0x7f591c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:05.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 --2- 172.21.15.47:0/795271164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1047a0 0x7f591c104c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f59040099c0 tx=0x7f5904009cd0 comp rx=0 tx=0).stop 2024-05-11T12:20:05.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 -- 172.21.15.47:0/795271164 shutdown_connections 2024-05-11T12:20:05.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 --2- 172.21.15.47:0/795271164 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f591c105140 0x7f591c10b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 --2- 172.21.15.47:0/795271164 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c1035a0 0x7f591c1039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 --2- 172.21.15.47:0/795271164 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1047a0 0x7f591c104c00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 -- 172.21.15.47:0/795271164 >> 172.21.15.47:0/795271164 conn(0x7f591c0feb40 msgr2=0x7f591c100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 -- 172.21.15.47:0/795271164 shutdown_connections 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.825+0000 7f5922e16700 1 -- 172.21.15.47:0/795271164 wait complete. 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 Processor -- start 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 -- start start 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1035a0 0x7f591c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f591c1047a0 0x7f591c19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:05.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c105140 0x7f591c1a2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f591c10f060 con 0x7f591c1035a0 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f591c10eee0 con 0x7f591c1047a0 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5922e16700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f591c10f1e0 con 0x7f591c105140 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1035a0 0x7f591c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1035a0 0x7f591c19b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37244/0 (socket says 172.21.15.47:37244) 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 -- 172.21.15.47:0/1134224848 learned_addr learned my addr 172.21.15.47:0/1134224848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f591bfff700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f591c1047a0 0x7f591c19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f59213b3700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c105140 0x7f591c1a2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 -- 172.21.15.47:0/1134224848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c105140 msgr2=0x7f591c1a2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c105140 0x7f591c1a2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 -- 172.21.15.47:0/1134224848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f591c1047a0 msgr2=0x7f591c19bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:05.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f591c1047a0 0x7f591c19bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.826+0000 7f5920bb2700 1 -- 172.21.15.47:0/1134224848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591c1a34f0 con 0x7f591c1035a0 2024-05-11T12:20:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.827+0000 7f5920bb2700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1035a0 0x7f591c19b890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f590c007e30 tx=0x7f590c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.827+0000 7f5919ffb700 1 -- 172.21.15.47:0/1134224848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c016020 con 0x7f591c1035a0 2024-05-11T12:20:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.827+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591c1a37e0 con 0x7f591c1035a0 2024-05-11T12:20:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.827+0000 7f5919ffb700 1 -- 172.21.15.47:0/1134224848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f590c00ad00 con 0x7f591c1035a0 2024-05-11T12:20:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.827+0000 7f5919ffb700 1 -- 172.21.15.47:0/1134224848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590c0115e0 con 0x7f591c1035a0 2024-05-11T12:20:05.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.827+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591c1a3d70 con 0x7f591c1035a0 2024-05-11T12:20:05.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.828+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591c06a240 con 0x7f591c1035a0 2024-05-11T12:20:05.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.829+0000 7f5919ffb700 1 -- 172.21.15.47:0/1134224848 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f590c019020 con 0x7f591c1035a0 2024-05-11T12:20:05.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.830+0000 7f5919ffb700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f59080b2880 0x7f59080b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:05.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.830+0000 7f591bfff700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f59080b2880 0x7f59080b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:05.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.830+0000 7f5919ffb700 1 -- 172.21.15.47:0/1134224848 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6847+0+0 (secure 0 0 0) 0x7f590c0e1920 con 0x7f591c1035a0 2024-05-11T12:20:05.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.831+0000 7f591bfff700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f59080b2880 0x7f59080b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5904009990 tx=0x7f5904003880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:05.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:05.835+0000 7f5919ffb700 1 -- 172.21.15.47:0/1134224848 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f590c020030 con 0x7f591c1035a0 2024-05-11T12:20:05.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:05 smithi064 ceph-mon[34537]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:20:05.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:05 smithi064 ceph-mon[34537]: osdmap e70: 12 total, 11 up, 12 in 2024-05-11T12:20:05.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:05 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:05.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:05 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:05.977 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:05 smithi064 ceph-mon[34537]: from='osd.11 ' entity='osd.11' 2024-05-11T12:20:06.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:05 smithi047 ceph-mon[30375]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:20:06.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:05 smithi047 ceph-mon[30375]: osdmap e70: 12 total, 11 up, 12 in 2024-05-11T12:20:06.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:05 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:06.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:05 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:06.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:05 smithi047 ceph-mon[30375]: from='osd.11 ' entity='osd.11' 2024-05-11T12:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.049+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f591c1a4630 con 0x7f591c1035a0 2024-05-11T12:20:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.049+0000 7f5919ffb700 1 -- 172.21.15.47:0/1134224848 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f590c0a9c00 con 0x7f591c1035a0 2024-05-11T12:20:06.049 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:06.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.051+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f59080b2880 msgr2=0x7f59080b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:06.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f59080b2880 0x7f59080b4d40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5904009990 tx=0x7f5904003880 comp rx=0 tx=0).stop 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1035a0 msgr2=0x7f591c19b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1035a0 0x7f591c19b890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f590c007e30 tx=0x7f590c00a340 comp rx=0 tx=0).stop 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 shutdown_connections 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f591c105140 0x7f591c1a2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f591c1035a0 0x7f591c19b890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f59080b2880 0x7f59080b4d40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 --2- 172.21.15.47:0/1134224848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f591c1047a0 0x7f591c19bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 >> 172.21.15.47:0/1134224848 conn(0x7f591c0feb40 msgr2=0x7f591c075780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 shutdown_connections 2024-05-11T12:20:06.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:06.052+0000 7f5922e16700 1 -- 172.21.15.47:0/1134224848 wait complete. 2024-05-11T12:20:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:05 smithi050 ceph-mon[34072]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi064", "root=default"]}]': finished 2024-05-11T12:20:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:05 smithi050 ceph-mon[34072]: osdmap e70: 12 total, 11 up, 12 in 2024-05-11T12:20:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:05 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:05 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:05 smithi050 ceph-mon[34072]: from='osd.11 ' entity='osd.11' 2024-05-11T12:20:06.848 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1715429979,"num_in_osds":12,"osd_in_since":1715429985,"num_remapped_pgs":0} 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: purged_snaps scrub starts 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: purged_snaps scrub ok 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1134224848' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] boot 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: osdmap e71: 12 total, 12 up, 12 in 2024-05-11T12:20:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: purged_snaps scrub starts 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: purged_snaps scrub ok 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1134224848' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] boot 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: osdmap e71: 12 total, 12 up, 12 in 2024-05-11T12:20:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:07.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: purged_snaps scrub starts 2024-05-11T12:20:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: purged_snaps scrub ok 2024-05-11T12:20:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-05-11T12:20:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1134224848' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T12:20:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: osd.11 [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] boot 2024-05-11T12:20:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: osdmap e71: 12 total, 12 up, 12 in 2024-05-11T12:20:07.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-11T12:20:07.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd stat -f json 2024-05-11T12:20:08.121 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: osdmap e72: 12 total, 12 up, 12 in 2024-05-11T12:20:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: Detected new or changed devices on smithi064 2024-05-11T12:20:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: osdmap e72: 12 total, 12 up, 12 in 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: Detected new or changed devices on smithi064 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-11T12:20:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:09.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: osdmap e72: 12 total, 12 up, 12 in 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: Detected new or changed devices on smithi064 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi064 to 4272M 2024-05-11T12:20:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:09.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:09.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:09.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:09 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:09.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:09 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:09.774 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:09 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 -- 172.21.15.47:0/3472085577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 msgr2=0x7efda01039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:09.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 --2- 172.21.15.47:0/3472085577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda01039f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0099c0 tx=0x7efd9c009cd0 comp rx=0 tx=0).stop 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 -- 172.21.15.47:0/3472085577 shutdown_connections 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 --2- 172.21.15.47:0/3472085577 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efda0105170 0x7efda010ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 --2- 172.21.15.47:0/3472085577 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda01039f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 --2- 172.21.15.47:0/3472085577 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda01047d0 0x7efda0104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 -- 172.21.15.47:0/3472085577 >> 172.21.15.47:0/3472085577 conn(0x7efda00feb70 msgr2=0x7efda0100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 -- 172.21.15.47:0/3472085577 shutdown_connections 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 -- 172.21.15.47:0/3472085577 wait complete. 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 Processor -- start 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.880+0000 7efda790f700 1 -- start start 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda790f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda00772f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda790f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efda01047d0 0x7efda0075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda790f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda0105170 0x7efda0075e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda790f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efda010f0c0 con 0x7efda01035d0 2024-05-11T12:20:09.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda790f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efda010ef40 con 0x7efda01047d0 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda790f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efda010f240 con 0x7efda0105170 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda00772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda00772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48916/0 (socket says 172.21.15.47:48916) 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 -- 172.21.15.47:0/1688253520 learned_addr learned my addr 172.21.15.47:0/1688253520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda4eaa700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efda01047d0 0x7efda0075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda5eac700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda0105170 0x7efda0075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 -- 172.21.15.47:0/1688253520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda0105170 msgr2=0x7efda0075e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda0105170 0x7efda0075e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 -- 172.21.15.47:0/1688253520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efda01047d0 msgr2=0x7efda0075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efda01047d0 0x7efda0075940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 -- 172.21.15.47:0/1688253520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda00763c0 con 0x7efda01035d0 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda5eac700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda0105170 0x7efda0075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda56ab700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda00772f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0038c0 tx=0x7efd9c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:09.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efd967fc700 1 -- 172.21.15.47:0/1688253520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c027070 con 0x7efda01035d0 2024-05-11T12:20:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.881+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda0076650 con 0x7efda01035d0 2024-05-11T12:20:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.882+0000 7efd967fc700 1 -- 172.21.15.47:0/1688253520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd9c004160 con 0x7efda01035d0 2024-05-11T12:20:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.882+0000 7efd967fc700 1 -- 172.21.15.47:0/1688253520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c022e20 con 0x7efda01035d0 2024-05-11T12:20:09.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.882+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda01a81c0 con 0x7efda01035d0 2024-05-11T12:20:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.882+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda006a240 con 0x7efda01035d0 2024-05-11T12:20:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.884+0000 7efd967fc700 1 -- 172.21.15.47:0/1688253520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efd9c02a070 con 0x7efda01035d0 2024-05-11T12:20:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.885+0000 7efd967fc700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd8c0b2880 0x7efd8c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.885+0000 7efda4eaa700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd8c0b2880 0x7efd8c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.885+0000 7efd967fc700 1 -- 172.21.15.47:0/1688253520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7efd9c0e2a40 con 0x7efda01035d0 2024-05-11T12:20:09.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.885+0000 7efda4eaa700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd8c0b2880 0x7efd8c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd90009a10 tx=0x7efd90009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:09.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:09.888+0000 7efd967fc700 1 -- 172.21.15.47:0/1688253520 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd9c013070 con 0x7efda01035d0 2024-05-11T12:20:10.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.102+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7efda0064160 con 0x7efda01035d0 2024-05-11T12:20:10.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.102+0000 7efd967fc700 1 -- 172.21.15.47:0/1688253520 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7efd9c0aabf0 con 0x7efda01035d0 2024-05-11T12:20:10.102 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:10.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.104+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd8c0b2880 msgr2=0x7efd8c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.104+0000 7efda790f700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd8c0b2880 0x7efd8c0b4d40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd90009a10 tx=0x7efd90009190 comp rx=0 tx=0).stop 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.104+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 msgr2=0x7efda00772f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda00772f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0038c0 tx=0x7efd9c0047b0 comp rx=0 tx=0).stop 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 shutdown_connections 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efda0105170 0x7efda0075e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efda01035d0 0x7efda00772f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd8c0b2880 0x7efd8c0b4d40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 --2- 172.21.15.47:0/1688253520 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efda01047d0 0x7efda0075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 >> 172.21.15.47:0/1688253520 conn(0x7efda00feb70 msgr2=0x7efda0107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 shutdown_connections 2024-05-11T12:20:10.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:10.105+0000 7efda790f700 1 -- 172.21.15.47:0/1688253520 wait complete. 2024-05-11T12:20:10.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:09 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:10.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:09 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:10.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:09 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:10.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:09 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:10.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:09 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:10.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:09 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:10.913 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715430006,"num_in_osds":12,"osd_in_since":1715429985,"num_remapped_pgs":0} 2024-05-11T12:20:10.914 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:20:11.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:10 smithi064 ceph-mon[34537]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:11.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:10 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1688253520' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T12:20:11.172 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:11.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:10 smithi050 ceph-mon[34072]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:11.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:10 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1688253520' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T12:20:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:10 smithi047 ceph-mon[30375]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:10 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1688253520' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-11T12:20:12.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 -- 172.21.15.47:0/110461963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102450 msgr2=0x7fc9041028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/110461963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102450 0x7fc9041028d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009990 tx=0x7fc8f4009ca0 comp rx=0 tx=0).stop 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 -- 172.21.15.47:0/110461963 shutdown_connections 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/110461963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904102e10 0x7fc90410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/110461963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc904108530 0x7fc904108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/110461963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102450 0x7fc9041028d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 -- 172.21.15.47:0/110461963 >> 172.21.15.47:0/110461963 conn(0x7fc904079400 msgr2=0x7fc904079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 -- 172.21.15.47:0/110461963 shutdown_connections 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.934+0000 7fc90b5c4700 1 -- 172.21.15.47:0/110461963 wait complete. 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 Processor -- start 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 -- start start 2024-05-11T12:20:12.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904102450 0x7fc90419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102e10 0x7fc90419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc904108530 0x7fc9041a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041125e0 con 0x7fc904102e10 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc904112460 con 0x7fc904108530 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc90b5c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc904112760 con 0x7fc904102450 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102e10 0x7fc90419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102e10 0x7fc90419af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48926/0 (socket says 172.21.15.47:48926) 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 -- 172.21.15.47:0/3407750785 learned_addr learned my addr 172.21.15.47:0/3407750785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc909360700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904102450 0x7fc90419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc909b61700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc904108530 0x7fc9041a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 -- 172.21.15.47:0/3407750785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904102450 msgr2=0x7fc90419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904102450 0x7fc90419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 -- 172.21.15.47:0/3407750785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc904108530 msgr2=0x7fc9041a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc904108530 0x7fc9041a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:12.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc908b5f700 1 -- 172.21.15.47:0/3407750785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041a6000 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.935+0000 7fc909b61700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc904108530 0x7fc9041a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.936+0000 7fc908b5f700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102e10 0x7fc90419af80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40057c0 tx=0x7fc8f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.936+0000 7fc8fa7fc700 1 -- 172.21.15.47:0/3407750785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4027070 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.936+0000 7fc8fa7fc700 1 -- 172.21.15.47:0/3407750785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8f4003f20 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.936+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9041a61d0 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.936+0000 7fc8fa7fc700 1 -- 172.21.15.47:0/3407750785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4022bc0 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.936+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041a6700 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.937+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90406a240 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.938+0000 7fc8fa7fc700 1 -- 172.21.15.47:0/3407750785 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc8f402a070 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.939+0000 7fc8fa7fc700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc8f00b2900 0x7fc8f00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.939+0000 7fc909360700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc8f00b2900 0x7fc8f00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.939+0000 7fc8fa7fc700 1 -- 172.21.15.47:0/3407750785 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc8f40e3400 con 0x7fc904102e10 2024-05-11T12:20:12.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.940+0000 7fc909360700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc8f00b2900 0x7fc8f00b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc900005950 tx=0x7fc90000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:12.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:12.943+0000 7fc8fa7fc700 1 -- 172.21.15.47:0/3407750785 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8f4031020 con 0x7fc904102e10 2024-05-11T12:20:13.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:12 smithi047 ceph-mon[30375]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:13.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:12 smithi064 ceph-mon[34537]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:13.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.162+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc90402e570 con 0x7fc904102e10 2024-05-11T12:20:13.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.163+0000 7fc8fa7fc700 1 -- 172.21.15.47:0/3407750785 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18348 (secure 0 0 0) 0x7fc8f4013070 con 0x7fc904102e10 2024-05-11T12:20:13.163 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:13.163 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":72,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:20:07.754921+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:20:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc8f00b2900 msgr2=0x7fc8f00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc8f00b2900 0x7fc8f00b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc900005950 tx=0x7fc90000b410 comp rx=0 tx=0).stop 2024-05-11T12:20:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102e10 msgr2=0x7fc90419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:13.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102e10 0x7fc90419af80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40057c0 tx=0x7fc8f40057f0 comp rx=0 tx=0).stop 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 shutdown_connections 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc8f00b2900 0x7fc8f00b4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc904108530 0x7fc9041a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc904102450 0x7fc90419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 --2- 172.21.15.47:0/3407750785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc904102e10 0x7fc90419af80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.165+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 >> 172.21.15.47:0/3407750785 conn(0x7fc904079400 msgr2=0x7fc904100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.166+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 shutdown_connections 2024-05-11T12:20:13.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:13.166+0000 7fc90b5c4700 1 -- 172.21.15.47:0/3407750785 wait complete. 2024-05-11T12:20:13.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:12 smithi050 ceph-mon[34072]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:13.908 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:16:11.517702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:20:13.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-11T12:20:14.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:13 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3407750785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:20:14.166 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:14.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:13 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3407750785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:20:14.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:13 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3407750785' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:20:15.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:14 smithi064 ceph-mon[34537]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:15.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:14 smithi050 ceph-mon[34072]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:14 smithi047 ceph-mon[30375]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:15.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.887+0000 7f98b56ba700 1 -- 172.21.15.47:0/1405240553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0108920 msgr2=0x7f98b01104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.887+0000 7f98b56ba700 1 --2- 172.21.15.47:0/1405240553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0108920 0x7f98b01104a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98a800b0d0 tx=0x7f98a800b3e0 comp rx=0 tx=0).stop 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.887+0000 7f98b56ba700 1 -- 172.21.15.47:0/1405240553 shutdown_connections 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.887+0000 7f98b56ba700 1 --2- 172.21.15.47:0/1405240553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0108920 0x7f98b01104a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.887+0000 7f98b56ba700 1 --2- 172.21.15.47:0/1405240553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b0103830 0x7f98b0103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.887+0000 7f98b56ba700 1 --2- 172.21.15.47:0/1405240553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98b01041e0 0x7f98b01081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.887+0000 7f98b56ba700 1 -- 172.21.15.47:0/1405240553 >> 172.21.15.47:0/1405240553 conn(0x7f98b007a6c0 msgr2=0x7f98b007aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.888+0000 7f98b56ba700 1 -- 172.21.15.47:0/1405240553 shutdown_connections 2024-05-11T12:20:15.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.888+0000 7f98b56ba700 1 -- 172.21.15.47:0/1405240553 wait complete. 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.888+0000 7f98b56ba700 1 Processor -- start 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.888+0000 7f98b56ba700 1 -- start start 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98b56ba700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0103830 0x7f98b019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98b56ba700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98b01041e0 0x7f98b019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98b56ba700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b0108920 0x7f98b0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98b56ba700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f98b01136f0 con 0x7f98b0103830 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98b56ba700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f98b0113570 con 0x7f98b01041e0 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98b56ba700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f98b0113870 con 0x7f98b0108920 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0103830 0x7f98b019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:15.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0103830 0x7f98b019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48948/0 (socket says 172.21.15.47:48948) 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 -- 172.21.15.47:0/3268157463 learned_addr learned my addr 172.21.15.47:0/3268157463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98ae7fc700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98b01041e0 0x7f98b019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98af7fe700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b0108920 0x7f98b0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 -- 172.21.15.47:0/3268157463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b0108920 msgr2=0x7f98b0199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b0108920 0x7f98b0199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 -- 172.21.15.47:0/3268157463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98b01041e0 msgr2=0x7f98b019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98b01041e0 0x7f98b019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 -- 172.21.15.47:0/3268157463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b019a090 con 0x7f98b0103830 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98aeffd700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0103830 0x7f98b019f770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98a000d980 tx=0x7f98a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f9897fff700 1 -- 172.21.15.47:0/3268157463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0016070 con 0x7f98b0103830 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f9897fff700 1 -- 172.21.15.47:0/3268157463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98a0004830 con 0x7f98b0103830 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b019a380 con 0x7f98b0103830 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.889+0000 7f9897fff700 1 -- 172.21.15.47:0/3268157463 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a0005540 con 0x7f98b0103830 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.890+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b01a6640 con 0x7f98b0103830 2024-05-11T12:20:15.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.890+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b006a240 con 0x7f98b0103830 2024-05-11T12:20:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.892+0000 7f9897fff700 1 -- 172.21.15.47:0/3268157463 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f98a0004220 con 0x7f98b0103830 2024-05-11T12:20:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.893+0000 7f9897fff700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f98980b2880 0x7f98980b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:15.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.893+0000 7f98ae7fc700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f98980b2880 0x7f98980b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.893+0000 7f9897fff700 1 -- 172.21.15.47:0/3268157463 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f98a00e2150 con 0x7f98b0103830 2024-05-11T12:20:15.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.893+0000 7f98ae7fc700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f98980b2880 0x7f98980b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98b0108350 tx=0x7f98a400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:15.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:15.896+0000 7f9897fff700 1 -- 172.21.15.47:0/3268157463 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98a001f020 con 0x7f98b0103830 2024-05-11T12:20:16.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.108+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98b019ae60 con 0x7f98b0103830 2024-05-11T12:20:16.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.109+0000 7f9897fff700 1 -- 172.21.15.47:0/3268157463 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f98a00cae20 con 0x7f98b0103830 2024-05-11T12:20:16.109 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-05-11T12:20:16.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f98980b2880 msgr2=0x7f98980b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f98980b2880 0x7f98980b4d40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f98b0108350 tx=0x7f98a400b410 comp rx=0 tx=0).stop 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0103830 msgr2=0x7f98b019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0103830 0x7f98b019f770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98a000d980 tx=0x7f98a000dc90 comp rx=0 tx=0).stop 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 shutdown_connections 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f98b0108920 0x7f98b0199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f98b0103830 0x7f98b019f770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f98980b2880 0x7f98980b4d40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 --2- 172.21.15.47:0/3268157463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f98b01041e0 0x7f98b019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 >> 172.21.15.47:0/3268157463 conn(0x7f98b007a6c0 msgr2=0x7f98b0102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 shutdown_connections 2024-05-11T12:20:16.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:16.111+0000 7f98b56ba700 1 -- 172.21.15.47:0/3268157463 wait complete. 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.a on smithi047 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.d on smithi047 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.g on smithi047 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.j on smithi047 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.b on smithi050 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.e on smithi050 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.h on smithi050 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.k on smithi050 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.c on smithi064 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.f on smithi064 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.i on smithi064 2024-05-11T12:20:16.814 INFO:tasks.cephadm:Adding mds.l on smithi064 2024-05-11T12:20:16.815 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph orch apply mds all '12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l' 2024-05-11T12:20:17.073 INFO:teuthology.orchestra.run.smithi064.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.c/config 2024-05-11T12:20:17.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:16 smithi064 ceph-mon[34537]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:17.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:16 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3268157463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:20:17.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:16 smithi050 ceph-mon[34072]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:17.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:16 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3268157463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:20:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:16 smithi047 ceph-mon[30375]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:16 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3268157463' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:20:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 -- 172.21.15.64:0/2698088841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c01035b0 msgr2=0x7f52c01039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 --2- 172.21.15.64:0/2698088841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c01035b0 0x7f52c01039d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009990 tx=0x7f52b0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:20:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 -- 172.21.15.64:0/2698088841 shutdown_connections 2024-05-11T12:20:18.899 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 --2- 172.21.15.64:0/2698088841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c0105150 0x7f52c010ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:18.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 --2- 172.21.15.64:0/2698088841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c01035b0 0x7f52c01039d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:18.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 --2- 172.21.15.64:0/2698088841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c01047b0 0x7f52c0104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:18.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 -- 172.21.15.64:0/2698088841 >> 172.21.15.64:0/2698088841 conn(0x7f52c00feb50 msgr2=0x7f52c0100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:18.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 -- 172.21.15.64:0/2698088841 shutdown_connections 2024-05-11T12:20:18.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.898+0000 7f52c605c700 1 -- 172.21.15.64:0/2698088841 wait complete. 2024-05-11T12:20:18.900 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 Processor -- start 2024-05-11T12:20:18.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 -- start start 2024-05-11T12:20:18.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c01035b0 0x7f52c019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:18.902 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c01047b0 0x7f52c019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c0105150 0x7f52c01a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010f050 con 0x7f52c01047b0 2024-05-11T12:20:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010eed0 con 0x7f52c01035b0 2024-05-11T12:20:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52c605c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010f1d0 con 0x7f52c0105150 2024-05-11T12:20:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c0105150 0x7f52c01a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c0105150 0x7f52c01a2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.64:60232/0 (socket says 172.21.15.64:60232) 2024-05-11T12:20:18.903 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 -- 172.21.15.64:0/3753399437 learned_addr learned my addr 172.21.15.64:0/3753399437 (peer_addr_for_me v2:172.21.15.64:0/0) 2024-05-11T12:20:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52beffd700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c01047b0 0x7f52c019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bf7fe700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c01035b0 0x7f52c019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 -- 172.21.15.64:0/3753399437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c01035b0 msgr2=0x7f52c019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c01035b0 0x7f52c019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 -- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c01047b0 msgr2=0x7f52c019bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c01047b0 0x7f52c019bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:18.904 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52bffff700 1 -- 172.21.15.64:0/3753399437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c01a35c0 con 0x7f52c0105150 2024-05-11T12:20:18.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.899+0000 7f52beffd700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c01047b0 0x7f52c019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:20:18.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.900+0000 7f52bffff700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c0105150 0x7f52c01a2f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52b800ef60 tx=0x7f52b800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:18.905 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.900+0000 7f52bcff9700 1 -- 172.21.15.64:0/3753399437 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b80092e0 con 0x7f52c0105150 2024-05-11T12:20:18.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.900+0000 7f52bcff9700 1 -- 172.21.15.64:0/3753399437 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b8004bb0 con 0x7f52c0105150 2024-05-11T12:20:18.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.900+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c01a3850 con 0x7f52c0105150 2024-05-11T12:20:18.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.900+0000 7f52bcff9700 1 -- 172.21.15.64:0/3753399437 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8004020 con 0x7f52c0105150 2024-05-11T12:20:18.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.900+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c01a3d90 con 0x7f52c0105150 2024-05-11T12:20:18.906 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.901+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c006a240 con 0x7f52c0105150 2024-05-11T12:20:18.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.902+0000 7f52bcff9700 1 -- 172.21.15.64:0/3753399437 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f52b801b030 con 0x7f52c0105150 2024-05-11T12:20:18.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.903+0000 7f52bcff9700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2900 0x7f52a80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:18.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.903+0000 7f52bcff9700 1 -- 172.21.15.64:0/3753399437 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f52b80e2150 con 0x7f52c0105150 2024-05-11T12:20:18.908 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.903+0000 7f52bf7fe700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2900 0x7f52a80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:18.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.904+0000 7f52bf7fe700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2900 0x7f52a80b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52b000b720 tx=0x7f52b0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:18.909 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:18.907+0000 7f52bcff9700 1 -- 172.21.15.64:0/3753399437 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f52b8022020 con 0x7f52c0105150 2024-05-11T12:20:19.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:18 smithi064 ceph-mon[34537]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:19.138 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.137+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}) -- 0x7f52c010b7f0 con 0x7f52a80b2900 2024-05-11T12:20:19.146 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.145+0000 7f52bcff9700 1 -- 172.21.15.64:0/3753399437 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f52c010b7f0 con 0x7f52a80b2900 2024-05-11T12:20:19.147 INFO:teuthology.orchestra.run.smithi064.stdout:Scheduled mds.all update... 2024-05-11T12:20:19.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2900 msgr2=0x7f52a80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:19.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2900 0x7f52a80b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52b000b720 tx=0x7f52b0005fd0 comp rx=0 tx=0).stop 2024-05-11T12:20:19.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c0105150 msgr2=0x7f52c01a2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:19.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c0105150 0x7f52c01a2f20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f52b800ef60 tx=0x7f52b800ef90 comp rx=0 tx=0).stop 2024-05-11T12:20:19.149 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 shutdown_connections 2024-05-11T12:20:19.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c0105150 0x7f52c01a2f20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:19.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2900 0x7f52a80b4dc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:19.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c01035b0 0x7f52c019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:19.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 --2- 172.21.15.64:0/3753399437 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c01047b0 0x7f52c019bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:19.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.147+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 >> 172.21.15.64:0/3753399437 conn(0x7f52c00feb50 msgr2=0x7f52c0109fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:19.150 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.148+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 shutdown_connections 2024-05-11T12:20:19.151 INFO:teuthology.orchestra.run.smithi064.stderr:2024-05-11T12:20:19.148+0000 7f52c605c700 1 -- 172.21.15.64:0/3753399437 wait complete. 2024-05-11T12:20:19.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:18 smithi050 ceph-mon[34072]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:19.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:18 smithi047 ceph-mon[30375]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:19.812 DEBUG:teuthology.orchestra.run.smithi047:mds.a> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.a.service 2024-05-11T12:20:19.816 DEBUG:teuthology.orchestra.run.smithi047:mds.d> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.d.service 2024-05-11T12:20:19.859 DEBUG:teuthology.orchestra.run.smithi047:mds.g> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.g.service 2024-05-11T12:20:19.903 DEBUG:teuthology.orchestra.run.smithi047:mds.j> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.j.service 2024-05-11T12:20:19.945 DEBUG:teuthology.orchestra.run.smithi050:mds.b> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.b.service 2024-05-11T12:20:19.948 DEBUG:teuthology.orchestra.run.smithi050:mds.e> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.e.service 2024-05-11T12:20:19.992 DEBUG:teuthology.orchestra.run.smithi050:mds.h> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.h.service 2024-05-11T12:20:20.035 DEBUG:teuthology.orchestra.run.smithi050:mds.k> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.k.service 2024-05-11T12:20:20.077 DEBUG:teuthology.orchestra.run.smithi064:mds.c> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.c.service 2024-05-11T12:20:20.080 DEBUG:teuthology.orchestra.run.smithi064:mds.f> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.f.service 2024-05-11T12:20:20.122 DEBUG:teuthology.orchestra.run.smithi064:mds.i> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.i.service 2024-05-11T12:20:20.166 DEBUG:teuthology.orchestra.run.smithi064:mds.l> sudo journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.l.service 2024-05-11T12:20:20.209 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-11T12:20:20.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:20.286 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:20:20.286 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: Deploying daemon mds.l on smithi064 2024-05-11T12:20:20.287 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:20 smithi064 ceph-mon[34537]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:20.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:20:20.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-11T12:20:20.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:20.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:20.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:20.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:20.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:20.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:20.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:20.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:20.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: Deploying daemon mds.l on smithi064 2024-05-11T12:20:20.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:20 smithi050 ceph-mon[34072]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:20.471 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:20:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: Saving service mds.all spec with placement smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-11T12:20:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:20:20.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:20.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:20.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:20.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:20.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: Deploying daemon mds.l on smithi064 2024-05-11T12:20:20.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:20 smithi047 ceph-mon[30375]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:21.614 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:21 smithi064 systemd[1]: Starting Ceph mds.l for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:22.030 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:21 smithi064 podman[65106]: 2024-05-11 12:20:21.644767399 +0000 UTC m=+0.019073002 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:22.030 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:21 smithi064 podman[65106]: 2024-05-11 12:20:21.768904719 +0000 UTC m=+0.143210349 container create 9be27e948c6c4e2b1953901b0cc1c80bf6e91971f0bf694aa8271ffcb141bcca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:20:22.031 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:22 smithi064 podman[65106]: 2024-05-11 12:20:22.030161051 +0000 UTC m=+0.404466642 container init 9be27e948c6c4e2b1953901b0cc1c80bf6e91971f0bf694aa8271ffcb141bcca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-11T12:20:22.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 -- 172.21.15.47:0/697959925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 msgr2=0x7f0e841081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:22.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 --2- 172.21.15.47:0/697959925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e841081f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0e780099c0 tx=0x7f0e78009cd0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 -- 172.21.15.47:0/697959925 shutdown_connections 2024-05-11T12:20:22.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 --2- 172.21.15.47:0/697959925 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e84108920 0x7f0e841104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 --2- 172.21.15.47:0/697959925 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e84103830 0x7f0e84103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 --2- 172.21.15.47:0/697959925 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e841081f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 -- 172.21.15.47:0/697959925 >> 172.21.15.47:0/697959925 conn(0x7f0e8407a6c0 msgr2=0x7f0e8407aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:22.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 -- 172.21.15.47:0/697959925 shutdown_connections 2024-05-11T12:20:22.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 -- 172.21.15.47:0/697959925 wait complete. 2024-05-11T12:20:22.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.232+0000 7f0e895cf700 1 Processor -- start 2024-05-11T12:20:22.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e895cf700 1 -- start start 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e895cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e84103830 0x7f0e84075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e895cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e84075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e895cf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e84108920 0x7f0e840762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e895cf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84113700 con 0x7f0e841041e0 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e895cf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84113580 con 0x7f0e84108920 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e895cf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0e84113880 con 0x7f0e84103830 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e84075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e84075d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51962/0 (socket says 172.21.15.47:51962) 2024-05-11T12:20:22.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 -- 172.21.15.47:0/399382649 learned_addr learned my addr 172.21.15.47:0/399382649 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:22.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e82ffd700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e84103830 0x7f0e84075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:22.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e837fe700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e84108920 0x7f0e840762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:22.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 -- 172.21.15.47:0/399382649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e84103830 msgr2=0x7f0e84075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:22.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e84103830 0x7f0e84075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 -- 172.21.15.47:0/399382649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e84108920 msgr2=0x7f0e840762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:22.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e84108920 0x7f0e840762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 -- 172.21.15.47:0/399382649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e841a80c0 con 0x7f0e841041e0 2024-05-11T12:20:22.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.233+0000 7f0e827fc700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e84075d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b580 tx=0x7f0e780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:22.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.234+0000 7f0e6bfff700 1 -- 172.21.15.47:0/399382649 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78027070 con 0x7f0e841041e0 2024-05-11T12:20:22.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.234+0000 7f0e6bfff700 1 -- 172.21.15.47:0/399382649 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e78003f20 con 0x7f0e841041e0 2024-05-11T12:20:22.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.234+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e841a8350 con 0x7f0e841041e0 2024-05-11T12:20:22.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.234+0000 7f0e6bfff700 1 -- 172.21.15.47:0/399382649 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e78022d10 con 0x7f0e841041e0 2024-05-11T12:20:22.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.234+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e841a87a0 con 0x7f0e841041e0 2024-05-11T12:20:22.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.235+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8410baf0 con 0x7f0e841041e0 2024-05-11T12:20:22.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.236+0000 7f0e6bfff700 1 -- 172.21.15.47:0/399382649 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f0e78022470 con 0x7f0e841041e0 2024-05-11T12:20:22.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.237+0000 7f0e6bfff700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:22.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.237+0000 7f0e82ffd700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:22.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.237+0000 7f0e6bfff700 1 -- 172.21.15.47:0/399382649 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f0e780e2d10 con 0x7f0e841041e0 2024-05-11T12:20:22.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.237+0000 7f0e82ffd700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005950 tx=0x7f0e7400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:22.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.240+0000 7f0e6bfff700 1 -- 172.21.15.47:0/399382649 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e78013070 con 0x7f0e841041e0 2024-05-11T12:20:22.364 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:22 smithi064 podman[65106]: 2024-05-11 12:20:22.033070353 +0000 UTC m=+0.407375951 container start 9be27e948c6c4e2b1953901b0cc1c80bf6e91971f0bf694aa8271ffcb141bcca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-11T12:20:22.364 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mds[65125]: -- 172.21.15.64:0/1021476917 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d9568fa40 con 0x555d95662800 2024-05-11T12:20:22.365 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l[65121]: starting mds.l at 2024-05-11T12:20:22.365 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:22 smithi064 bash[65106]: 9be27e948c6c4e2b1953901b0cc1c80bf6e91971f0bf694aa8271ffcb141bcca 2024-05-11T12:20:22.365 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:20:22 smithi064 systemd[1]: Started Ceph mds.l for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.512+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e8402e570 con 0x7f0e841041e0 2024-05-11T12:20:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.513+0000 7f0e6bfff700 1 -- 172.21.15.47:0/399382649 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1390 (secure 0 0 0) 0x7f0e780aaf60 con 0x7f0e841041e0 2024-05-11T12:20:22.514 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:22.514 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.515+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0e6c0b2880 msgr2=0x7f0e6c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.515+0000 7f0e895cf700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e74005950 tx=0x7f0e7400a300 comp rx=0 tx=0).stop 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.515+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 msgr2=0x7f0e84075d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.515+0000 7f0e895cf700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e84075d90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800b580 tx=0x7f0e780047b0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.515+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 shutdown_connections 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.516+0000 7f0e895cf700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0e6c0b2880 0x7f0e6c0b4d40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.516+0000 7f0e895cf700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0e84108920 0x7f0e840762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.516+0000 7f0e895cf700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0e84103830 0x7f0e84075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.516+0000 7f0e895cf700 1 --2- 172.21.15.47:0/399382649 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0e841041e0 0x7f0e84075d90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:22.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.516+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 >> 172.21.15.47:0/399382649 conn(0x7f0e8407a6c0 msgr2=0x7f0e84101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.516+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 shutdown_connections 2024-05-11T12:20:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:22.516+0000 7f0e895cf700 1 -- 172.21.15.47:0/399382649 wait complete. 2024-05-11T12:20:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 2 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.64:6832/395408259,v1:172.21.15.64:6833/395408259] up:boot 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: fsmap 1 up:standby 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:22.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:22.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:22.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:22 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/399382649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.64:6832/395408259,v1:172.21.15.64:6833/395408259] up:boot 2024-05-11T12:20:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: fsmap 1 up:standby 2024-05-11T12:20:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T12:20:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:22.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:22.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:22.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:22 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/399382649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.64:6832/395408259,v1:172.21.15.64:6833/395408259] up:boot 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: fsmap 1 up:standby 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:22.900 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:22.901 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:22 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/399382649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:23.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:23 smithi064 ceph-mon[34537]: Deploying daemon mds.k on smithi050 2024-05-11T12:20:23.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:23 smithi050 ceph-mon[34072]: Deploying daemon mds.k on smithi050 2024-05-11T12:20:24.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:23 smithi047 ceph-mon[30375]: Deploying daemon mds.k on smithi050 2024-05-11T12:20:24.580 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:24 smithi050 systemd[1]: Starting Ceph mds.k for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:24.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:24 smithi064 ceph-mon[34537]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:24.877 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:24 smithi050 ceph-mon[34072]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:24.877 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:24 smithi050 podman[63507]: 2024-05-11 12:20:24.629595841 +0000 UTC m=+0.018862131 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:24.877 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:24 smithi050 podman[63507]: 2024-05-11 12:20:24.75392562 +0000 UTC m=+0.143191876 container create 77aa021b21a68582af448a1962fc252ddae60f071046683b875df4e4b5cadf61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-05-11T12:20:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:24 smithi047 ceph-mon[30375]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:25.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:25.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:24 smithi050 podman[63507]: 2024-05-11 12:20:24.992073771 +0000 UTC m=+0.381340033 container init 77aa021b21a68582af448a1962fc252ddae60f071046683b875df4e4b5cadf61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507) 2024-05-11T12:20:25.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:24 smithi050 podman[63507]: 2024-05-11 12:20:24.994963818 +0000 UTC m=+0.384230082 container start 77aa021b21a68582af448a1962fc252ddae60f071046683b875df4e4b5cadf61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-11T12:20:25.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:25 smithi050 ceph-mds[63526]: -- 172.21.15.50:0/1965705157 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f2088dfa40 con 0x55f2088b3000 2024-05-11T12:20:25.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:25 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k[63522]: starting mds.k at 2024-05-11T12:20:25.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:25 smithi050 bash[63507]: 77aa021b21a68582af448a1962fc252ddae60f071046683b875df4e4b5cadf61 2024-05-11T12:20:25.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:20:25 smithi050 systemd[1]: Started Ceph mds.k for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:25.414 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:26.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:26.412 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:26.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:26.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: Deploying daemon mds.j on smithi047 2024-05-11T12:20:26.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:26.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.50:6832/995235547,v1:172.21.15.50:6833/995235547] up:boot 2024-05-11T12:20:26.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: fsmap 2 up:standby 2024-05-11T12:20:26.413 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:26 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T12:20:26.432 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: Deploying daemon mds.j on smithi047 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.50:6832/995235547,v1:172.21.15.50:6833/995235547] up:boot 2024-05-11T12:20:26.433 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: fsmap 2 up:standby 2024-05-11T12:20:26.434 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:26 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: Deploying daemon mds.j on smithi047 2024-05-11T12:20:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:26.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.50:6832/995235547,v1:172.21.15.50:6833/995235547] up:boot 2024-05-11T12:20:26.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: fsmap 2 up:standby 2024-05-11T12:20:26.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:26 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 -- 172.21.15.47:0/1694551226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 msgr2=0x7f9bc81104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1694551226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 0x7f9bc81104a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f9bc000b0a0 tx=0x7f9bc000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 -- 172.21.15.47:0/1694551226 shutdown_connections 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1694551226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 0x7f9bc81104a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1694551226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bc8103830 0x7f9bc8103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1694551226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bc81041e0 0x7f9bc81081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 -- 172.21.15.47:0/1694551226 >> 172.21.15.47:0/1694551226 conn(0x7f9bc807a6c0 msgr2=0x7f9bc807aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 -- 172.21.15.47:0/1694551226 shutdown_connections 2024-05-11T12:20:28.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.304+0000 7f9bd0321700 1 -- 172.21.15.47:0/1694551226 wait complete. 2024-05-11T12:20:28.307 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:28 smithi047 systemd[1]: Starting Ceph mds.j for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.305+0000 7f9bd0321700 1 Processor -- start 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.305+0000 7f9bd0321700 1 -- start start 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.305+0000 7f9bd0321700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bc8103830 0x7f9bc819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.305+0000 7f9bd0321700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bc81041e0 0x7f9bc819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bd0321700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 0x7f9bc8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bd0321700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8113700 con 0x7f9bc81041e0 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bcd8bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bc81041e0 0x7f9bc819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bcd8bc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bc81041e0 0x7f9bc819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40910/0 (socket says 172.21.15.47:40910) 2024-05-11T12:20:28.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bcd8bc700 1 -- 172.21.15.47:0/1736200409 learned_addr learned my addr 172.21.15.47:0/1736200409 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8113580 con 0x7f9bc8103830 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bce0bd700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bc8103830 0x7f9bc819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc8113880 con 0x7f9bc8108920 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bce8be700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 0x7f9bc8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bce0bd700 1 -- 172.21.15.47:0/1736200409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 msgr2=0x7f9bc8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bce0bd700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 0x7f9bc8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bce0bd700 1 -- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bc81041e0 msgr2=0x7f9bc819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:28.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bce0bd700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bc81041e0 0x7f9bc819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.306+0000 7f9bce0bd700 1 -- 172.21.15.47:0/1736200409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc819a0c0 con 0x7f9bc8103830 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.307+0000 7f9bce0bd700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bc8103830 0x7f9bc819f7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400e9f0 tx=0x7f9bc400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.307+0000 7f9bbb7fe700 1 -- 172.21.15.47:0/1736200409 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc400c9c0 con 0x7f9bc8103830 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.307+0000 7f9bbb7fe700 1 -- 172.21.15.47:0/1736200409 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc400cb60 con 0x7f9bc8103830 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.307+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc819a3b0 con 0x7f9bc8103830 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.307+0000 7f9bbb7fe700 1 -- 172.21.15.47:0/1736200409 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4018680 con 0x7f9bc8103830 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.307+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc81a6670 con 0x7f9bc8103830 2024-05-11T12:20:28.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.308+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc806a240 con 0x7f9bc8103830 2024-05-11T12:20:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.310+0000 7f9bbb7fe700 1 -- 172.21.15.47:0/1736200409 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9bc4010430 con 0x7f9bc8103830 2024-05-11T12:20:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.311+0000 7f9bbb7fe700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9bb40b2900 0x7f9bb40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.311+0000 7f9bcd8bc700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9bb40b2900 0x7f9bb40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.311+0000 7f9bbb7fe700 1 -- 172.21.15.47:0/1736200409 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f9bc40e22e0 con 0x7f9bc8103830 2024-05-11T12:20:28.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.311+0000 7f9bcd8bc700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9bb40b2900 0x7f9bb40b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9bc81082c0 tx=0x7f9bbc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:28.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.314+0000 7f9bbb7fe700 1 -- 172.21.15.47:0/1736200409 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9bc40aa400 con 0x7f9bc8103830 2024-05-11T12:20:28.574 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:28 smithi047 podman[69315]: 2024-05-11 12:20:28.419313681 +0000 UTC m=+0.023042205 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:28.575 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:28 smithi047 podman[69315]: 2024-05-11 12:20:28.52339635 +0000 UTC m=+0.127124859 container create f44a52a5393e311676ddeebe3ce09c4b80b6b4ffb4860f87bb24b7546a5928cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-11T12:20:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.600+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc802e570 con 0x7f9bc8103830 2024-05-11T12:20:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.601+0000 7f9bbb7fe700 1 -- 172.21.15.47:0/1736200409 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2212 (secure 0 0 0) 0x7f9bc40a9d00 con 0x7f9bc8103830 2024-05-11T12:20:28.602 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:28.602 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T12:20:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9bb40b2900 msgr2=0x7f9bb40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9bb40b2900 0x7f9bb40b4dc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9bc81082c0 tx=0x7f9bbc009260 comp rx=0 tx=0).stop 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bc8103830 msgr2=0x7f9bc819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bc8103830 0x7f9bc819f7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9bc400e9f0 tx=0x7f9bc400ed00 comp rx=0 tx=0).stop 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 shutdown_connections 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9bc8108920 0x7f9bc8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9bb40b2900 0x7f9bb40b4dc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9bc8103830 0x7f9bc819f7a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 --2- 172.21.15.47:0/1736200409 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9bc81041e0 0x7f9bc819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 >> 172.21.15.47:0/1736200409 conn(0x7f9bc807a6c0 msgr2=0x7f9bc8102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 shutdown_connections 2024-05-11T12:20:28.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:28.604+0000 7f9bd0321700 1 -- 172.21.15.47:0/1736200409 wait complete. 2024-05-11T12:20:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-05-11T12:20:28.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:28 smithi064 ceph-mon[34537]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:28.935 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:28 smithi047 ceph-mon[30375]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:28 smithi050 ceph-mon[34072]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:29.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:28 smithi047 podman[69315]: 2024-05-11 12:20:28.936221471 +0000 UTC m=+0.539949994 container init f44a52a5393e311676ddeebe3ce09c4b80b6b4ffb4860f87bb24b7546a5928cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:20:29.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:28 smithi047 podman[69315]: 2024-05-11 12:20:28.939802302 +0000 UTC m=+0.543530811 container start f44a52a5393e311676ddeebe3ce09c4b80b6b4ffb4860f87bb24b7546a5928cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-11T12:20:29.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:28 smithi047 ceph-mds[69347]: -- 172.21.15.47:0/3667608150 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556bafa53a40 con 0x556bafa26c00 2024-05-11T12:20:29.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:28 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j[69338]: starting mds.j at 2024-05-11T12:20:29.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:29 smithi047 bash[69315]: f44a52a5393e311676ddeebe3ce09c4b80b6b4ffb4860f87bb24b7546a5928cb 2024-05-11T12:20:29.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:20:29 smithi047 systemd[1]: Started Ceph mds.j for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:29.763 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:29 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1736200409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:29.763 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.763 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.764 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.764 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:29.764 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:29.764 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:29 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1736200409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:29.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:29 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1736200409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:30 smithi047 ceph-mon[30375]: Deploying daemon mds.d on smithi047 2024-05-11T12:20:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:30 smithi047 ceph-mon[30375]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:30 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.47:6834/3289515287,v1:172.21.15.47:6835/3289515287] up:boot 2024-05-11T12:20:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:30 smithi047 ceph-mon[30375]: fsmap 3 up:standby 2024-05-11T12:20:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:30 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T12:20:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:30 smithi064 ceph-mon[34537]: Deploying daemon mds.d on smithi047 2024-05-11T12:20:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:30 smithi064 ceph-mon[34537]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:30 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.47:6834/3289515287,v1:172.21.15.47:6835/3289515287] up:boot 2024-05-11T12:20:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:30 smithi064 ceph-mon[34537]: fsmap 3 up:standby 2024-05-11T12:20:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:30 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T12:20:30.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:30 smithi050 ceph-mon[34072]: Deploying daemon mds.d on smithi047 2024-05-11T12:20:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:30 smithi050 ceph-mon[34072]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:30 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.47:6834/3289515287,v1:172.21.15.47:6835/3289515287] up:boot 2024-05-11T12:20:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:30 smithi050 ceph-mon[34072]: fsmap 3 up:standby 2024-05-11T12:20:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:30 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-11T12:20:31.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:31.487 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 systemd[1]: Starting Ceph mds.d for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:31.761 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 podman[69638]: 2024-05-11 12:20:31.390399138 +0000 UTC m=+0.019042893 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:31.761 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 podman[69638]: 2024-05-11 12:20:31.509026009 +0000 UTC m=+0.137669764 container create 060ade8db75e7d2f6d685e4ac688ff891e6617693bddfb1b6bd729abc08c7c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507) 2024-05-11T12:20:31.761 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 podman[69638]: 2024-05-11 12:20:31.762188847 +0000 UTC m=+0.390832605 container init 060ade8db75e7d2f6d685e4ac688ff891e6617693bddfb1b6bd729abc08c7c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:20:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:32.025 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 podman[69638]: 2024-05-11 12:20:31.765588796 +0000 UTC m=+0.394232560 container start 060ade8db75e7d2f6d685e4ac688ff891e6617693bddfb1b6bd729abc08c7c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-11T12:20:32.025 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 ceph-mds[69689]: -- 172.21.15.47:0/202377903 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1cb977a40 con 0x55a1cb94b000 2024-05-11T12:20:32.025 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d[69671]: starting mds.d at 2024-05-11T12:20:32.025 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 bash[69638]: 060ade8db75e7d2f6d685e4ac688ff891e6617693bddfb1b6bd729abc08c7c64 2024-05-11T12:20:32.025 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:20:31 smithi047 systemd[1]: Started Ceph mds.d for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: Deploying daemon mds.e on smithi050 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.47:6836/2341314538,v1:172.21.15.47:6837/2341314538] up:boot 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: fsmap 4 up:standby 2024-05-11T12:20:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:32 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: Deploying daemon mds.e on smithi050 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.47:6836/2341314538,v1:172.21.15.47:6837/2341314538] up:boot 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: fsmap 4 up:standby 2024-05-11T12:20:33.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:32 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T12:20:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:33.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: Deploying daemon mds.e on smithi050 2024-05-11T12:20:33.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.47:6836/2341314538,v1:172.21.15.47:6837/2341314538] up:boot 2024-05-11T12:20:33.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: fsmap 4 up:standby 2024-05-11T12:20:33.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:32 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-11T12:20:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 -- 172.21.15.47:0/4261859999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c103830 msgr2=0x7f802c103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 --2- 172.21.15.47:0/4261859999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c103830 0x7f802c103c10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b0d0 tx=0x7f801c00b3e0 comp rx=0 tx=0).stop 2024-05-11T12:20:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 -- 172.21.15.47:0/4261859999 shutdown_connections 2024-05-11T12:20:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 --2- 172.21.15.47:0/4261859999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c108920 0x7f802c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 --2- 172.21.15.47:0/4261859999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c103830 0x7f802c103c10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:33.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 --2- 172.21.15.47:0/4261859999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c1041e0 0x7f802c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 -- 172.21.15.47:0/4261859999 >> 172.21.15.47:0/4261859999 conn(0x7f802c07a6c0 msgr2=0x7f802c07aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 -- 172.21.15.47:0/4261859999 shutdown_connections 2024-05-11T12:20:33.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.816+0000 7f8030fde700 1 -- 172.21.15.47:0/4261859999 wait complete. 2024-05-11T12:20:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 Processor -- start 2024-05-11T12:20:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 -- start start 2024-05-11T12:20:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c103830 0x7f802c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c1041e0 0x7f802c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c108920 0x7f802c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f802c113720 con 0x7f802c1041e0 2024-05-11T12:20:33.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1135a0 con 0x7f802c108920 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8030fde700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f802c1138a0 con 0x7f802c103830 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8029d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c1041e0 0x7f802c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8029d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c1041e0 0x7f802c19fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40974/0 (socket says 172.21.15.47:40974) 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f8029d9b700 1 -- 172.21.15.47:0/3017819553 learned_addr learned my addr 172.21.15.47:0/3017819553 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f802ad9d700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c108920 0x7f802c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.817+0000 7f802a59c700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c103830 0x7f802c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8029d9b700 1 -- 172.21.15.47:0/3017819553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c103830 msgr2=0x7f802c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8029d9b700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c103830 0x7f802c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8029d9b700 1 -- 172.21.15.47:0/3017819553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c108920 msgr2=0x7f802c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8029d9b700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c108920 0x7f802c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8029d9b700 1 -- 172.21.15.47:0/3017819553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c19a160 con 0x7f802c1041e0 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8029d9b700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c1041e0 0x7f802c19fd80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f802000d980 tx=0x7f802000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f801b7fe700 1 -- 172.21.15.47:0/3017819553 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020017070 con 0x7f802c1041e0 2024-05-11T12:20:33.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f801b7fe700 1 -- 172.21.15.47:0/3017819553 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8020005020 con 0x7f802c1041e0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c19a450 con 0x7f802c1041e0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f801b7fe700 1 -- 172.21.15.47:0/3017819553 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802000fa10 con 0x7f802c1041e0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.818+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c1a6710 con 0x7f802c1041e0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.819+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802c10bb60 con 0x7f802c1041e0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.821+0000 7f801b7fe700 1 -- 172.21.15.47:0/3017819553 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8020004a30 con 0x7f802c1041e0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.821+0000 7f801b7fe700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80140b2880 0x7f80140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.822+0000 7f802a59c700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80140b2880 0x7f80140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.822+0000 7f801b7fe700 1 -- 172.21.15.47:0/3017819553 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f80200e2040 con 0x7f802c1041e0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.822+0000 7f802a59c700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80140b2880 0x7f80140b4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f801c009fd0 tx=0x7f801c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:33.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:33.825+0000 7f801b7fe700 1 -- 172.21.15.47:0/3017819553 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8020020020 con 0x7f802c1041e0 2024-05-11T12:20:34.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.086+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802c06a240 con 0x7f802c1041e0 2024-05-11T12:20:34.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.087+0000 7f801b7fe700 1 -- 172.21.15.47:0/3017819553 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7f80200aa200 con 0x7f802c1041e0 2024-05-11T12:20:34.087 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:34.087 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80140b2880 msgr2=0x7f80140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80140b2880 0x7f80140b4d40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f801c009fd0 tx=0x7f801c00b040 comp rx=0 tx=0).stop 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c1041e0 msgr2=0x7f802c19fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c1041e0 0x7f802c19fd80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f802000d980 tx=0x7f802000dc90 comp rx=0 tx=0).stop 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 shutdown_connections 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80140b2880 0x7f80140b4d40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f802c108920 0x7f802c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f802c103830 0x7f802c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 --2- 172.21.15.47:0/3017819553 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802c1041e0 0x7f802c19fd80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:34.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 >> 172.21.15.47:0/3017819553 conn(0x7f802c07a6c0 msgr2=0x7f802c109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:34.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 shutdown_connections 2024-05-11T12:20:34.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:34.089+0000 7f8030fde700 1 -- 172.21.15.47:0/3017819553 wait complete. 2024-05-11T12:20:34.090 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 5 2024-05-11T12:20:34.403 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 systemd[1]: Starting Ceph mds.e for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:34.671 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 podman[63827]: 2024-05-11 12:20:34.307122614 +0000 UTC m=+0.018586910 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:34.671 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 podman[63827]: 2024-05-11 12:20:34.418384497 +0000 UTC m=+0.129848769 container create 230b1ad531e2f327c1489d769e6dc6854be9bf8d4fc2b3a4d7d03b490cdf95e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:20:34.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3017819553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:34.949 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 podman[63827]: 2024-05-11 12:20:34.671616366 +0000 UTC m=+0.383080640 container init 230b1ad531e2f327c1489d769e6dc6854be9bf8d4fc2b3a4d7d03b490cdf95e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:20:34.950 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 podman[63827]: 2024-05-11 12:20:34.674411638 +0000 UTC m=+0.385875910 container start 230b1ad531e2f327c1489d769e6dc6854be9bf8d4fc2b3a4d7d03b490cdf95e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-11T12:20:34.950 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 ceph-mds[63845]: -- 172.21.15.50:0/1879015337 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ebd241a40 con 0x561ebd214800 2024-05-11T12:20:34.950 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e[63841]: starting mds.e at 2024-05-11T12:20:34.950 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 bash[63827]: 230b1ad531e2f327c1489d769e6dc6854be9bf8d4fc2b3a4d7d03b490cdf95e2 2024-05-11T12:20:34.950 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:20:34 smithi050 systemd[1]: Started Ceph mds.e for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3017819553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3017819553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:35.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:35.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:35.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:35.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:36.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:35 smithi064 ceph-mon[34537]: Deploying daemon mds.f on smithi064 2024-05-11T12:20:36.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:35 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.50:6834/1314438872,v1:172.21.15.50:6835/1314438872] up:boot 2024-05-11T12:20:36.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:35 smithi064 ceph-mon[34537]: fsmap 5 up:standby 2024-05-11T12:20:36.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:35 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T12:20:36.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:35 smithi050 ceph-mon[34072]: Deploying daemon mds.f on smithi064 2024-05-11T12:20:36.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:35 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.50:6834/1314438872,v1:172.21.15.50:6835/1314438872] up:boot 2024-05-11T12:20:36.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:35 smithi050 ceph-mon[34072]: fsmap 5 up:standby 2024-05-11T12:20:36.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:35 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T12:20:36.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:35 smithi047 ceph-mon[30375]: Deploying daemon mds.f on smithi064 2024-05-11T12:20:36.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:35 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.50:6834/1314438872,v1:172.21.15.50:6835/1314438872] up:boot 2024-05-11T12:20:36.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:35 smithi047 ceph-mon[30375]: fsmap 5 up:standby 2024-05-11T12:20:36.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:35 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T12:20:36.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:37.038 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:37.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:36 smithi064 ceph-mon[34537]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:37.114 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 systemd[1]: Starting Ceph mds.f for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:37.161 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:36 smithi047 ceph-mon[30375]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:37.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:36 smithi050 ceph-mon[34072]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:37.492 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 podman[65414]: 2024-05-11 12:20:37.231332437 +0000 UTC m=+0.113948887 container create 8014ca27afe382545cd44266e55622e0630a8991ae02b8569800194ca82523fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-11T12:20:37.492 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 podman[65414]: 2024-05-11 12:20:37.136092782 +0000 UTC m=+0.018709252 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:37.864 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 podman[65414]: 2024-05-11 12:20:37.592303114 +0000 UTC m=+0.474919564 container init 8014ca27afe382545cd44266e55622e0630a8991ae02b8569800194ca82523fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:20:37.864 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 podman[65414]: 2024-05-11 12:20:37.595190381 +0000 UTC m=+0.477806832 container start 8014ca27afe382545cd44266e55622e0630a8991ae02b8569800194ca82523fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-11T12:20:37.864 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 ceph-mds[65432]: -- 172.21.15.64:0/3473224880 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b6e485a40 con 0x561b6e459000 2024-05-11T12:20:37.864 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f[65428]: starting mds.f at 2024-05-11T12:20:37.864 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 bash[65414]: 8014ca27afe382545cd44266e55622e0630a8991ae02b8569800194ca82523fd 2024-05-11T12:20:37.864 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:20:37 smithi064 systemd[1]: Started Ceph mds.f for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:38.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 -- 172.21.15.47:0/2906818225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 msgr2=0x7fad2006b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 --2- 172.21.15.47:0/2906818225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2006b6b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad14009990 tx=0x7fad14009ca0 comp rx=0 tx=0).stop 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 -- 172.21.15.47:0/2906818225 shutdown_connections 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 --2- 172.21.15.47:0/2906818225 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad2006bbf0 0x7fad201103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 --2- 172.21.15.47:0/2906818225 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad201095c0 0x7fad201099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 --2- 172.21.15.47:0/2906818225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2006b6b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 -- 172.21.15.47:0/2906818225 >> 172.21.15.47:0/2906818225 conn(0x7fad20079400 msgr2=0x7fad20079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 -- 172.21.15.47:0/2906818225 shutdown_connections 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.786+0000 7fad27c63700 1 -- 172.21.15.47:0/2906818225 wait complete. 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 Processor -- start 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 -- start start 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad2006bbf0 0x7fad2019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:38.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad201095c0 0x7fad20199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fad201136d0 con 0x7fad2006b230 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fad20113550 con 0x7fad2006bbf0 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad27c63700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fad20113850 con 0x7fad201095c0 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53838/0 (socket says 172.21.15.47:53838) 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 -- 172.21.15.47:0/2569901463 learned_addr learned my addr 172.21.15.47:0/2569901463 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad251fe700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad2006bbf0 0x7fad2019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad26200700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad201095c0 0x7fad20199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 -- 172.21.15.47:0/2569901463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad201095c0 msgr2=0x7fad20199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad201095c0 0x7fad20199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 -- 172.21.15.47:0/2569901463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad2006bbf0 msgr2=0x7fad2019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad2006bbf0 0x7fad2019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 -- 172.21.15.47:0/2569901463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2019a190 con 0x7fad2006b230 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad251fe700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad2006bbf0 0x7fad2019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:20:38.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.787+0000 7fad259ff700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2019f870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d950 tx=0x7fad1c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.788+0000 7fad12ffd700 1 -- 172.21.15.47:0/2569901463 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c016070 con 0x7fad2006b230 2024-05-11T12:20:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.788+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2019a480 con 0x7fad2006b230 2024-05-11T12:20:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.788+0000 7fad12ffd700 1 -- 172.21.15.47:0/2569901463 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1c010460 con 0x7fad2006b230 2024-05-11T12:20:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.788+0000 7fad12ffd700 1 -- 172.21.15.47:0/2569901463 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c00f620 con 0x7fad2006b230 2024-05-11T12:20:38.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.788+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad201a6740 con 0x7fad2006b230 2024-05-11T12:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.789+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2006a240 con 0x7fad2006b230 2024-05-11T12:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.790+0000 7fad12ffd700 1 -- 172.21.15.47:0/2569901463 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fad1c019070 con 0x7fad2006b230 2024-05-11T12:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.791+0000 7fad12ffd700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fad0c0b2880 0x7fad0c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.791+0000 7fad251fe700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fad0c0b2880 0x7fad0c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.791+0000 7fad12ffd700 1 -- 172.21.15.47:0/2569901463 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fad1c0e1d10 con 0x7fad2006b230 2024-05-11T12:20:38.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.791+0000 7fad251fe700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fad0c0b2880 0x7fad0c0b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad14006010 tx=0x7fad1401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:38.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:38.794+0000 7fad12ffd700 1 -- 172.21.15.47:0/2569901463 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fad1c0a9ed0 con 0x7fad2006b230 2024-05-11T12:20:38.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:38.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:38.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:38.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:38.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:38.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:38.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:38.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: Deploying daemon mds.c on smithi064 2024-05-11T12:20:38.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.64:6834/470230978,v1:172.21.15.64:6835/470230978] up:boot 2024-05-11T12:20:38.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: fsmap 6 up:standby 2024-05-11T12:20:38.795 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:38 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T12:20:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.062+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad2002e570 con 0x7fad2006b230 2024-05-11T12:20:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.063+0000 7fad12ffd700 1 -- 172.21.15.47:0/2569901463 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5509 (secure 0 0 0) 0x7fad1c0a97d0 con 0x7fad2006b230 2024-05-11T12:20:39.063 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:39.063 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-11T12:20:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.065+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fad0c0b2880 msgr2=0x7fad0c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.065+0000 7fad27c63700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fad0c0b2880 0x7fad0c0b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad14006010 tx=0x7fad1401a040 comp rx=0 tx=0).stop 2024-05-11T12:20:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.065+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 msgr2=0x7fad2019f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.065+0000 7fad27c63700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2019f870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00d950 tx=0x7fad1c00dd10 comp rx=0 tx=0).stop 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 shutdown_connections 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fad201095c0 0x7fad20199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad2006b230 0x7fad2019f870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fad0c0b2880 0x7fad0c0b4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 --2- 172.21.15.47:0/2569901463 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fad2006bbf0 0x7fad2019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 >> 172.21.15.47:0/2569901463 conn(0x7fad20079400 msgr2=0x7fad201011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 shutdown_connections 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:39.066+0000 7fad27c63700 1 -- 172.21.15.47:0/2569901463 wait complete. 2024-05-11T12:20:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-05-11T12:20:39.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:39.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:39.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:39.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:39.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:39.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:39.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:39.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: Deploying daemon mds.c on smithi064 2024-05-11T12:20:39.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.64:6834/470230978,v1:172.21.15.64:6835/470230978] up:boot 2024-05-11T12:20:39.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: fsmap 6 up:standby 2024-05-11T12:20:39.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:38 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T12:20:39.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: Deploying daemon mds.c on smithi064 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.64:6834/470230978,v1:172.21.15.64:6835/470230978] up:boot 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: fsmap 6 up:standby 2024-05-11T12:20:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:38 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-11T12:20:40.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:39 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2569901463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:40.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:39 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2569901463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:40.114 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:39 smithi064 systemd[1]: Starting Ceph mds.c for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:40.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:39 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2569901463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:40.456 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 podman[65723]: 2024-05-11 12:20:40.106487136 +0000 UTC m=+0.018238821 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:40.456 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 podman[65723]: 2024-05-11 12:20:40.220539599 +0000 UTC m=+0.132291325 container create ace55ea90732ef846b6b7221273d6bbf781127f6acb1bcc3f1ffec28a0451110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-11T12:20:40.456 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 podman[65723]: 2024-05-11 12:20:40.456564711 +0000 UTC m=+0.368316392 container init ace55ea90732ef846b6b7221273d6bbf781127f6acb1bcc3f1ffec28a0451110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:20:40.753 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 podman[65723]: 2024-05-11 12:20:40.459401163 +0000 UTC m=+0.371152846 container start ace55ea90732ef846b6b7221273d6bbf781127f6acb1bcc3f1ffec28a0451110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:20:40.754 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mds[65741]: -- 172.21.15.64:0/1721266209 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a61caada40 con 0x55a61ca80c00 2024-05-11T12:20:40.754 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c[65737]: starting mds.c at 2024-05-11T12:20:40.754 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 bash[65723]: ace55ea90732ef846b6b7221273d6bbf781127f6acb1bcc3f1ffec28a0451110 2024-05-11T12:20:40.754 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:20:40 smithi064 systemd[1]: Started Ceph mds.c for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:40 smithi047 ceph-mon[30375]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:40 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:41.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mon[34537]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:40 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:41.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:40 smithi050 ceph-mon[34072]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:40 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:41.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:41 smithi050 ceph-mon[34072]: Deploying daemon mds.h on smithi050 2024-05-11T12:20:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:41 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.64:6836/2460224166,v1:172.21.15.64:6837/2460224166] up:boot 2024-05-11T12:20:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:41 smithi050 ceph-mon[34072]: fsmap 7 up:standby 2024-05-11T12:20:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T12:20:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:41 smithi047 ceph-mon[30375]: Deploying daemon mds.h on smithi050 2024-05-11T12:20:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:41 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.64:6836/2460224166,v1:172.21.15.64:6837/2460224166] up:boot 2024-05-11T12:20:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:41 smithi047 ceph-mon[30375]: fsmap 7 up:standby 2024-05-11T12:20:42.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T12:20:42.037 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:41 smithi064 ceph-mon[34537]: Deploying daemon mds.h on smithi050 2024-05-11T12:20:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:41 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.64:6836/2460224166,v1:172.21.15.64:6837/2460224166] up:boot 2024-05-11T12:20:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:41 smithi064 ceph-mon[34537]: fsmap 7 up:standby 2024-05-11T12:20:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-11T12:20:43.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:42 smithi047 ceph-mon[30375]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:42 smithi064 ceph-mon[34537]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:43.153 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:42 smithi050 ceph-mon[34072]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:43.153 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:42 smithi050 systemd[1]: Starting Ceph mds.h for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:43.437 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 podman[64137]: 2024-05-11 12:20:43.056446335 +0000 UTC m=+0.018388467 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:43.438 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 podman[64137]: 2024-05-11 12:20:43.17654772 +0000 UTC m=+0.138489824 container create 7e9c270d0ce37bb7d202828b28dbfe20b98c514103da2ff76f48a1fcfeaea3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:20:43.438 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 podman[64137]: 2024-05-11 12:20:43.438428683 +0000 UTC m=+0.400370792 container init 7e9c270d0ce37bb7d202828b28dbfe20b98c514103da2ff76f48a1fcfeaea3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True) 2024-05-11T12:20:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/2055995672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102e40 msgr2=0x7fa6b010f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/2055995672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102e40 0x7fa6b010f300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800b0a0 tx=0x7fa6a800b3b0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/2055995672 shutdown_connections 2024-05-11T12:20:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/2055995672 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102e40 0x7fa6b010f300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/2055995672 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b01084c0 0x7fa6b01088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/2055995672 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b0102480 0x7fa6b0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/2055995672 >> 172.21.15.47:0/2055995672 conn(0x7fa6b0079400 msgr2=0x7fa6b0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/2055995672 shutdown_connections 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/2055995672 wait complete. 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.692+0000 7fa6b7e6b700 1 Processor -- start 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b7e6b700 1 -- start start 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b7e6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102480 0x7fa6b0104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b7e6b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0102e40 0x7fa6b01045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b7e6b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b01084c0 0x7fa6b0104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b7e6b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b01125d0 con 0x7fa6b0102480 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b7e6b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0112450 con 0x7fa6b01084c0 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b7e6b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b0112750 con 0x7fa6b0102e40 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5406700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0102e40 0x7fa6b01045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102480 0x7fa6b0104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:43.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102480 0x7fa6b0104090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53852/0 (socket says 172.21.15.47:53852) 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 -- 172.21.15.47:0/3520846524 learned_addr learned my addr 172.21.15.47:0/3520846524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b6408700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b01084c0 0x7fa6b0104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 -- 172.21.15.47:0/3520846524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0102e40 msgr2=0x7fa6b01045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0102e40 0x7fa6b01045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 -- 172.21.15.47:0/3520846524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b01084c0 msgr2=0x7fa6b0104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b01084c0 0x7fa6b0104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b5c07700 1 -- 172.21.15.47:0/3520846524 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b01a80b0 con 0x7fa6b0102480 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.693+0000 7fa6b6408700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b01084c0 0x7fa6b0104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.694+0000 7fa6b5c07700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102480 0x7fa6b0104090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c920 tx=0x7fa6ac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.694+0000 7fa6a6ffd700 1 -- 172.21.15.47:0/3520846524 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac004a30 con 0x7fa6b0102480 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.694+0000 7fa6a6ffd700 1 -- 172.21.15.47:0/3520846524 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ac0054d0 con 0x7fa6b0102480 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.694+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b01a83a0 con 0x7fa6b0102480 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.694+0000 7fa6a6ffd700 1 -- 172.21.15.47:0/3520846524 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac007c70 con 0x7fa6b0102480 2024-05-11T12:20:43.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.694+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b01a88e0 con 0x7fa6b0102480 2024-05-11T12:20:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.695+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b006a240 con 0x7fa6b0102480 2024-05-11T12:20:43.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.697+0000 7fa6a6ffd700 1 -- 172.21.15.47:0/3520846524 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa6ac004c60 con 0x7fa6b0102480 2024-05-11T12:20:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.698+0000 7fa6a6ffd700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa69c0b2880 0x7fa69c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.698+0000 7fa6b5406700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa69c0b2880 0x7fa69c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.698+0000 7fa6a6ffd700 1 -- 172.21.15.47:0/3520846524 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fa6ac0e1d80 con 0x7fa6b0102480 2024-05-11T12:20:43.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.699+0000 7fa6b5406700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa69c0b2880 0x7fa69c0b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0079ac0 tx=0x7fa6a00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:43.699 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 podman[64137]: 2024-05-11 12:20:43.441101163 +0000 UTC m=+0.403043269 container start 7e9c270d0ce37bb7d202828b28dbfe20b98c514103da2ff76f48a1fcfeaea3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-11T12:20:43.699 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 ceph-mds[64156]: -- 172.21.15.50:0/3488651128 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585bf275a40 con 0x5585bf249000 2024-05-11T12:20:43.699 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h[64152]: starting mds.h at 2024-05-11T12:20:43.699 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 bash[64137]: 7e9c270d0ce37bb7d202828b28dbfe20b98c514103da2ff76f48a1fcfeaea3aa 2024-05-11T12:20:43.700 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:20:43 smithi050 systemd[1]: Started Ceph mds.h for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:43.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.702+0000 7fa6a6ffd700 1 -- 172.21.15.47:0/3520846524 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6ac014030 con 0x7fa6b0102480 2024-05-11T12:20:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.968+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6b002e570 con 0x7fa6b0102480 2024-05-11T12:20:43.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.969+0000 7fa6a6ffd700 1 -- 172.21.15.47:0/3520846524 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7159 (secure 0 0 0) 0x7fa6ac0a9f40 con 0x7fa6b0102480 2024-05-11T12:20:43.969 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:43.969 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-11T12:20:43.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa69c0b2880 msgr2=0x7fa69c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:43.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa69c0b2880 0x7fa69c0b4d40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0079ac0 tx=0x7fa6a00073d0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102480 msgr2=0x7fa6b0104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102480 0x7fa6b0104090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00c920 tx=0x7fa6ac00cc30 comp rx=0 tx=0).stop 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 shutdown_connections 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa69c0b2880 0x7fa69c0b4d40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6b01084c0 0x7fa6b0104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6b0102480 0x7fa6b0104090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 --2- 172.21.15.47:0/3520846524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6b0102e40 0x7fa6b01045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.971+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 >> 172.21.15.47:0/3520846524 conn(0x7fa6b0079400 msgr2=0x7fa6b006bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.972+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 shutdown_connections 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:43.972+0000 7fa6b7e6b700 1 -- 172.21.15.47:0/3520846524 wait complete. 2024-05-11T12:20:43.972 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: Deploying daemon mds.i on smithi064 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] up:boot 2024-05-11T12:20:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: fsmap 8 up:standby 2024-05-11T12:20:44.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T12:20:44.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:44 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3520846524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:44.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: Deploying daemon mds.i on smithi064 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] up:boot 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: fsmap 8 up:standby 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T12:20:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:44 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3520846524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: Deploying daemon mds.i on smithi064 2024-05-11T12:20:45.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] up:boot 2024-05-11T12:20:45.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: fsmap 8 up:standby 2024-05-11T12:20:45.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T12:20:45.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:44 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3520846524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:46.114 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:45 smithi064 systemd[1]: Starting Ceph mds.i for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:46.386 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 podman[66031]: 2024-05-11 12:20:46.020091197 +0000 UTC m=+0.018528999 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:46.386 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 podman[66031]: 2024-05-11 12:20:46.143028191 +0000 UTC m=+0.141466038 container create 0ebc0df130aa4762a88fa9af8ffb65d805c6159149df89fb148b6c70fbd5139c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:20:46.386 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 podman[66031]: 2024-05-11 12:20:46.387171075 +0000 UTC m=+0.385608877 container init 0ebc0df130aa4762a88fa9af8ffb65d805c6159149df89fb148b6c70fbd5139c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-11T12:20:46.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mon[34537]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:46.864 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 podman[66031]: 2024-05-11 12:20:46.390025252 +0000 UTC m=+0.388463056 container start 0ebc0df130aa4762a88fa9af8ffb65d805c6159149df89fb148b6c70fbd5139c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-11T12:20:46.865 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 ceph-mds[66050]: -- 172.21.15.64:0/4182180415 <== mon.1 v2:172.21.15.64:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5596f2207a40 con 0x5596f21da800 2024-05-11T12:20:46.865 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i[66046]: starting mds.i at 2024-05-11T12:20:46.865 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 bash[66031]: 0ebc0df130aa4762a88fa9af8ffb65d805c6159149df89fb148b6c70fbd5139c 2024-05-11T12:20:46.865 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:20:46 smithi064 systemd[1]: Started Ceph mds.i for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:46.940 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:46 smithi050 ceph-mon[34072]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:47.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:46 smithi047 ceph-mon[30375]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:47.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:47.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:47.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:47.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:47.121 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:47.122 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:47.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:47 smithi047 ceph-mon[30375]: Deploying daemon mds.g on smithi047 2024-05-11T12:20:47.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:47 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.64:6838/4099525243,v1:172.21.15.64:6839/4099525243] up:boot 2024-05-11T12:20:47.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:47 smithi047 ceph-mon[30375]: fsmap 9 up:standby 2024-05-11T12:20:47.765 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T12:20:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:47 smithi064 ceph-mon[34537]: Deploying daemon mds.g on smithi047 2024-05-11T12:20:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:47 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.64:6838/4099525243,v1:172.21.15.64:6839/4099525243] up:boot 2024-05-11T12:20:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:47 smithi064 ceph-mon[34537]: fsmap 9 up:standby 2024-05-11T12:20:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T12:20:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:47 smithi050 ceph-mon[34072]: Deploying daemon mds.g on smithi047 2024-05-11T12:20:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:47 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.64:6838/4099525243,v1:172.21.15.64:6839/4099525243] up:boot 2024-05-11T12:20:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:47 smithi050 ceph-mon[34072]: fsmap 9 up:standby 2024-05-11T12:20:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-11T12:20:48.740 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:48 smithi047 ceph-mon[30375]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:48.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 -- 172.21.15.47:0/3854651182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c102480 msgr2=0x7fe72c102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 --2- 172.21.15.47:0/3854651182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c102480 0x7fe72c102900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe714009990 tx=0x7fe714009ca0 comp rx=0 tx=0).stop 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 -- 172.21.15.47:0/3854651182 shutdown_connections 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 --2- 172.21.15.47:0/3854651182 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe72c102e40 0x7fe72c10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 --2- 172.21.15.47:0/3854651182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c1084c0 0x7fe72c1088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 --2- 172.21.15.47:0/3854651182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c102480 0x7fe72c102900 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 -- 172.21.15.47:0/3854651182 >> 172.21.15.47:0/3854651182 conn(0x7fe72c079400 msgr2=0x7fe72c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 -- 172.21.15.47:0/3854651182 shutdown_connections 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.848+0000 7fe7325f3700 1 -- 172.21.15.47:0/3854651182 wait complete. 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 Processor -- start 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 -- start start 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c102480 0x7fe72c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe72c102e40 0x7fe72c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:48.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c1084c0 0x7fe72c1a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c112650 con 0x7fe72c1084c0 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1124d0 con 0x7fe72c102480 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe7325f3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe72c1127d0 con 0x7fe72c102e40 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.849+0000 7fe730b90700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c1084c0 0x7fe72c1a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c1084c0 0x7fe72c1a5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49444/0 (socket says 172.21.15.47:49444) 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 -- 172.21.15.47:0/624109781 learned_addr learned my addr 172.21.15.47:0/624109781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe723fff700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe72c102e40 0x7fe72c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe72bfff700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c102480 0x7fe72c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 -- 172.21.15.47:0/624109781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe72c102e40 msgr2=0x7fe72c19af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe72c102e40 0x7fe72c19af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 -- 172.21.15.47:0/624109781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c102480 msgr2=0x7fe72c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c102480 0x7fe72c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:48.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 -- 172.21.15.47:0/624109781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72c1a5ff0 con 0x7fe72c1084c0 2024-05-11T12:20:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe730b90700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c1084c0 0x7fe72c1a5ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe72400b970 tx=0x7fe72400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe729ffb700 1 -- 172.21.15.47:0/624109781 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724019070 con 0x7fe72c1084c0 2024-05-11T12:20:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72c1a6220 con 0x7fe72c1084c0 2024-05-11T12:20:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe729ffb700 1 -- 172.21.15.47:0/624109781 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe72400cde0 con 0x7fe72c1084c0 2024-05-11T12:20:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe729ffb700 1 -- 172.21.15.47:0/624109781 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe724007370 con 0x7fe72c1084c0 2024-05-11T12:20:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.850+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72c1a66e0 con 0x7fe72c1084c0 2024-05-11T12:20:48.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.851+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe72c103b70 con 0x7fe72c1084c0 2024-05-11T12:20:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.853+0000 7fe729ffb700 1 -- 172.21.15.47:0/624109781 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe72401c070 con 0x7fe72c1084c0 2024-05-11T12:20:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.853+0000 7fe729ffb700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe7180b2880 0x7fe7180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:48.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.854+0000 7fe72bfff700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe7180b2880 0x7fe7180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.854+0000 7fe729ffb700 1 -- 172.21.15.47:0/624109781 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fe7240e19d0 con 0x7fe72c1084c0 2024-05-11T12:20:48.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.854+0000 7fe72bfff700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe7180b2880 0x7fe7180b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005ea0 tx=0x7fe71c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:48.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:48.857+0000 7fe729ffb700 1 -- 172.21.15.47:0/624109781 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe724023070 con 0x7fe72c1084c0 2024-05-11T12:20:48.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:48 smithi064 ceph-mon[34537]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:48 smithi050 ceph-mon[34072]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:49.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.128+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe72c02e570 con 0x7fe72c1084c0 2024-05-11T12:20:49.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.129+0000 7fe729ffb700 1 -- 172.21.15.47:0/624109781 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7986 (secure 0 0 0) 0x7fe7240a9b90 con 0x7fe72c1084c0 2024-05-11T12:20:49.129 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:49.130 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-11T12:20:49.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe7180b2880 msgr2=0x7fe7180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe7180b2880 0x7fe7180b4d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe71c005ea0 tx=0x7fe71c005e30 comp rx=0 tx=0).stop 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c1084c0 msgr2=0x7fe72c1a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c1084c0 0x7fe72c1a5ab0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe72400b970 tx=0x7fe72400bc80 comp rx=0 tx=0).stop 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 shutdown_connections 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe72c1084c0 0x7fe72c1a5ab0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe7180b2880 0x7fe7180b4d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe72c102480 0x7fe72c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 --2- 172.21.15.47:0/624109781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe72c102e40 0x7fe72c19af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.131+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 >> 172.21.15.47:0/624109781 conn(0x7fe72c079400 msgr2=0x7fe72c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.132+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 shutdown_connections 2024-05-11T12:20:49.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:49.132+0000 7fe7325f3700 1 -- 172.21.15.47:0/624109781 wait complete. 2024-05-11T12:20:49.133 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 10 2024-05-11T12:20:49.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:49 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/624109781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:49.897 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:49 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/624109781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:49.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:49 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/624109781' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:50.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:50 smithi064 ceph-mon[34537]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:50.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:50 smithi050 ceph-mon[34072]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:51.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:50 smithi047 ceph-mon[30375]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:51.025 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:50 smithi047 systemd[1]: Starting Ceph mds.g for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:51.331 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 podman[70584]: 2024-05-11 12:20:50.952010435 +0000 UTC m=+0.018196410 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:51.331 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 podman[70584]: 2024-05-11 12:20:51.079338064 +0000 UTC m=+0.145524025 container create 573e378133e2ce8945570bd74778100a7f2784bd8e15b1e00d2edbb9f93a85de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:20:51.332 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 podman[70584]: 2024-05-11 12:20:51.332204872 +0000 UTC m=+0.398390842 container init 573e378133e2ce8945570bd74778100a7f2784bd8e15b1e00d2edbb9f93a85de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, ceph=True) 2024-05-11T12:20:51.775 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 podman[70584]: 2024-05-11 12:20:51.335365681 +0000 UTC m=+0.401551650 container start 573e378133e2ce8945570bd74778100a7f2784bd8e15b1e00d2edbb9f93a85de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD) 2024-05-11T12:20:51.775 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 ceph-mds[70603]: -- 172.21.15.47:0/1127359433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a8022ffa40 con 0x55a8022d2c00 2024-05-11T12:20:51.775 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g[70599]: starting mds.g at 2024-05-11T12:20:51.775 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 bash[70584]: 573e378133e2ce8945570bd74778100a7f2784bd8e15b1e00d2edbb9f93a85de 2024-05-11T12:20:51.775 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:20:51 smithi047 systemd[1]: Started Ceph mds.g for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:51.932 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: Deploying daemon mds.a on smithi047 2024-05-11T12:20:52.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:52.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.47:6838/1816706003,v1:172.21.15.47:6839/1816706003] up:boot 2024-05-11T12:20:52.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: fsmap 10 up:standby 2024-05-11T12:20:52.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T12:20:52.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:52.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:52.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:52.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: Deploying daemon mds.a on smithi047 2024-05-11T12:20:52.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:52.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.47:6838/1816706003,v1:172.21.15.47:6839/1816706003] up:boot 2024-05-11T12:20:52.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: fsmap 10 up:standby 2024-05-11T12:20:52.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: Deploying daemon mds.a on smithi047 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:52.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.47:6838/1816706003,v1:172.21.15.47:6839/1816706003] up:boot 2024-05-11T12:20:52.951 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: fsmap 10 up:standby 2024-05-11T12:20:52.951 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-11T12:20:53.207 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:53.587 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:53 smithi047 systemd[1]: Starting Ceph mds.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:20:54.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:54 smithi047 ceph-mon[30375]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:54.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:54 smithi064 ceph-mon[34537]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:54.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:54 smithi050 ceph-mon[34072]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:55.025 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:54 smithi047 podman[71010]: 2024-05-11 12:20:54.764399113 +0000 UTC m=+0.098921742 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:55.025 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:54 smithi047 podman[71010]: 2024-05-11 12:20:54.889085771 +0000 UTC m=+0.223608429 container create 1790197da40d80370a7659a871d59b28081aa655e16c159451464b2710867c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507) 2024-05-11T12:20:55.412 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:55 smithi047 podman[71010]: 2024-05-11 12:20:55.108528139 +0000 UTC m=+0.443050758 container init 1790197da40d80370a7659a871d59b28081aa655e16c159451464b2710867c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-11T12:20:55.412 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:55 smithi047 podman[71010]: 2024-05-11 12:20:55.111509703 +0000 UTC m=+0.446032325 container start 1790197da40d80370a7659a871d59b28081aa655e16c159451464b2710867c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:20:55.412 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:55 smithi047 ceph-mds[71033]: -- 172.21.15.47:0/3735293219 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56242895ba40 con 0x56242892e800 2024-05-11T12:20:55.412 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a[71029]: starting mds.a at 2024-05-11T12:20:55.412 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:55 smithi047 bash[71010]: 1790197da40d80370a7659a871d59b28081aa655e16c159451464b2710867c80 2024-05-11T12:20:55.412 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:20:55 smithi047 systemd[1]: Started Ceph mds.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 -- 172.21.15.47:0/278291585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf8102730 msgr2=0x7fcdf8102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:55.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 --2- 172.21.15.47:0/278291585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf8102730 0x7fcdf8102bb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009990 tx=0x7fcdec009ca0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 -- 172.21.15.47:0/278291585 shutdown_connections 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 --2- 172.21.15.47:0/278291585 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf81030f0 0x7fcdf810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 --2- 172.21.15.47:0/278291585 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdf8107470 0x7fcdf8107850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 --2- 172.21.15.47:0/278291585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf8102730 0x7fcdf8102bb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 -- 172.21.15.47:0/278291585 >> 172.21.15.47:0/278291585 conn(0x7fcdf8079310 msgr2=0x7fcdf8079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 -- 172.21.15.47:0/278291585 shutdown_connections 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.415+0000 7fcdfd792700 1 -- 172.21.15.47:0/278291585 wait complete. 2024-05-11T12:20:55.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.416+0000 7fcdfd792700 1 Processor -- start 2024-05-11T12:20:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.416+0000 7fcdfd792700 1 -- start start 2024-05-11T12:20:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdfd792700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdf8102730 0x7fcdf8199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdfd792700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf81030f0 0x7fcdf819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdfd792700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf8107470 0x7fcdf819e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdfd792700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81127b0 con 0x7fcdf81030f0 2024-05-11T12:20:55.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdfd792700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8112630 con 0x7fcdf8102730 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdfd792700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8112930 con 0x7fcdf8107470 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf81030f0 0x7fcdf819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf81030f0 0x7fcdf819a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49506/0 (socket says 172.21.15.47:49506) 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdf8102730 0x7fcdf8199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 -- 172.21.15.47:0/2559882664 learned_addr learned my addr 172.21.15.47:0/2559882664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf77fe700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf8107470 0x7fcdf819e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 -- 172.21.15.47:0/2559882664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf8107470 msgr2=0x7fcdf819e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf8107470 0x7fcdf819e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 -- 172.21.15.47:0/2559882664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdf8102730 msgr2=0x7fcdf8199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdf8102730 0x7fcdf8199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 -- 172.21.15.47:0/2559882664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf819eda0 con 0x7fcdf81030f0 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf6ffd700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdf8102730 0x7fcdf8199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.417+0000 7fcdf67fc700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf81030f0 0x7fcdf819a320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.418+0000 7fcddffff700 1 -- 172.21.15.47:0/2559882664 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec027070 con 0x7fcdf81030f0 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.418+0000 7fcddffff700 1 -- 172.21.15.47:0/2559882664 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec003f20 con 0x7fcdf81030f0 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.418+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf819f030 con 0x7fcdf81030f0 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.418+0000 7fcddffff700 1 -- 172.21.15.47:0/2559882664 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec022ca0 con 0x7fcdf81030f0 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.418+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf81a6910 con 0x7fcdf81030f0 2024-05-11T12:20:55.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.419+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf8103bd0 con 0x7fcdf81030f0 2024-05-11T12:20:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.420+0000 7fcddffff700 1 -- 172.21.15.47:0/2559882664 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcdec02a070 con 0x7fcdf81030f0 2024-05-11T12:20:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.421+0000 7fcddffff700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcde00b2880 0x7fcde00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:20:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.421+0000 7fcdf6ffd700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcde00b2880 0x7fcde00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:20:55.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.421+0000 7fcddffff700 1 -- 172.21.15.47:0/2559882664 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fcdec0e2b80 con 0x7fcdf81030f0 2024-05-11T12:20:55.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.421+0000 7fcdf6ffd700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcde00b2880 0x7fcde00b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009de0 tx=0x7fcde8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:20:55.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.425+0000 7fcddffff700 1 -- 172.21.15.47:0/2559882664 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcdec013070 con 0x7fcdf81030f0 2024-05-11T12:20:55.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.685+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdf806a240 con 0x7fcdf81030f0 2024-05-11T12:20:55.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.686+0000 7fcddffff700 1 -- 172.21.15.47:0/2559882664 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9638 (secure 0 0 0) 0x7fcdec0aadd0 con 0x7fcdf81030f0 2024-05-11T12:20:55.687 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:20:55.687 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-11T12:20:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.689+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcde00b2880 msgr2=0x7fcde00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.689+0000 7fcdfd792700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcde00b2880 0x7fcde00b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009de0 tx=0x7fcde8009190 comp rx=0 tx=0).stop 2024-05-11T12:20:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.689+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf81030f0 msgr2=0x7fcdf819a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:20:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.689+0000 7fcdfd792700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf81030f0 0x7fcdf819a320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcdec000c00 tx=0x7fcdec0047b0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.689+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 shutdown_connections 2024-05-11T12:20:55.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.690+0000 7fcdfd792700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdf8107470 0x7fcdf819e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.690+0000 7fcdfd792700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcde00b2880 0x7fcde00b4d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.690+0000 7fcdfd792700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdf8102730 0x7fcdf8199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.690+0000 7fcdfd792700 1 --2- 172.21.15.47:0/2559882664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdf81030f0 0x7fcdf819a320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:20:55.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.690+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 >> 172.21.15.47:0/2559882664 conn(0x7fcdf8079310 msgr2=0x7fcdf806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:20:55.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.690+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 shutdown_connections 2024-05-11T12:20:55.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:20:55.690+0000 7fcdfd792700 1 -- 172.21.15.47:0/2559882664 wait complete. 2024-05-11T12:20:55.692 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 12 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: Deploying daemon mds.b on smithi050 2024-05-11T12:20:56.348 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:56.349 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.47:6840/3463326526,v1:172.21.15.47:6841/3463326526] up:boot 2024-05-11T12:20:56.349 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: fsmap 11 up:standby 2024-05-11T12:20:56.349 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T12:20:56.349 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:56 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2559882664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: Deploying daemon mds.b on smithi050 2024-05-11T12:20:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:56.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.47:6840/3463326526,v1:172.21.15.47:6841/3463326526] up:boot 2024-05-11T12:20:56.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: fsmap 11 up:standby 2024-05-11T12:20:56.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T12:20:56.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:56 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2559882664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: Deploying daemon mds.b on smithi050 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.47:6840/3463326526,v1:172.21.15.47:6841/3463326526] up:boot 2024-05-11T12:20:56.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: fsmap 11 up:standby 2024-05-11T12:20:56.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-11T12:20:56.615 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:56 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2559882664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:20:57.950 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:57 smithi050 podman[64459]: 2024-05-11 12:20:57.733087742 +0000 UTC m=+0.017948519 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:20:58.257 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:57 smithi050 podman[64459]: 2024-05-11 12:20:57.955740194 +0000 UTC m=+0.240600986 container create 2ccb6b21283749d4973b32ce72ddbb30a8e7bef42fbddfe7168edc863bbda233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True) 2024-05-11T12:20:58.258 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:58 smithi050 podman[64459]: 2024-05-11 12:20:58.158458699 +0000 UTC m=+0.443319450 container init 2ccb6b21283749d4973b32ce72ddbb30a8e7bef42fbddfe7168edc863bbda233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-11T12:20:58.258 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:58 smithi050 podman[64459]: 2024-05-11 12:20:58.161043184 +0000 UTC m=+0.445903935 container start 2ccb6b21283749d4973b32ce72ddbb30a8e7bef42fbddfe7168edc863bbda233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-11T12:20:58.258 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mds[64478]: -- 172.21.15.50:0/2042490587 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ed672c3a40 con 0x55ed67296c00 2024-05-11T12:20:58.258 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b[64474]: starting mds.b at 2024-05-11T12:20:58.258 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:58 smithi050 bash[64459]: 2ccb6b21283749d4973b32ce72ddbb30a8e7bef42fbddfe7168edc863bbda233 2024-05-11T12:20:58.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:20:58.585 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:20:58 smithi050 systemd[1]: Started Ceph mds.b for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:20:58.649 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:20:58.772 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:58 smithi047 ceph-mon[30375]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:58.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.773 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:58 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:58 smithi064 ceph-mon[34537]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:58 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:58.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mon[34072]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:20:58.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:20:58.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:58 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:20:59.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:59 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.50:6838/1725807153,v1:172.21.15.50:6839/1725807153] up:boot 2024-05-11T12:20:59.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:59 smithi064 ceph-mon[34537]: fsmap 12 up:standby 2024-05-11T12:20:59.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:20:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T12:20:59.916 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:59 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.50:6838/1725807153,v1:172.21.15.50:6839/1725807153] up:boot 2024-05-11T12:20:59.916 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:59 smithi047 ceph-mon[30375]: fsmap 12 up:standby 2024-05-11T12:20:59.917 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:20:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T12:20:59.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:59 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.50:6838/1725807153,v1:172.21.15.50:6839/1725807153] up:boot 2024-05-11T12:20:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:59 smithi050 ceph-mon[34072]: fsmap 12 up:standby 2024-05-11T12:20:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:20:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-11T12:21:00.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.328+0000 7fbde332e700 1 -- 172.21.15.47:0/1924647614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 msgr2=0x7fbddc102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 --2- 172.21.15.47:0/1924647614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc102e70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80099c0 tx=0x7fbdd8009cd0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 -- 172.21.15.47:0/1924647614 shutdown_connections 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 --2- 172.21.15.47:0/1924647614 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc1077e0 0x7fbddc10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 --2- 172.21.15.47:0/1924647614 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc102e70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 --2- 172.21.15.47:0/1924647614 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbddc103440 0x7fbddc1070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 -- 172.21.15.47:0/1924647614 >> 172.21.15.47:0/1924647614 conn(0x7fbddc079400 msgr2=0x7fbddc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 -- 172.21.15.47:0/1924647614 shutdown_connections 2024-05-11T12:21:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.329+0000 7fbde332e700 1 -- 172.21.15.47:0/1924647614 wait complete. 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 Processor -- start 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 -- start start 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbddc103440 0x7fbddc19b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc1077e0 0x7fbddc1a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc112640 con 0x7fbddc102a90 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1124c0 con 0x7fbddc103440 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde332e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbddc1127c0 con 0x7fbddc1077e0 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde10ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:00.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde18cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc1077e0 0x7fbddc1a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde10ca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56834/0 (socket says 172.21.15.47:56834) 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde10ca700 1 -- 172.21.15.47:0/2002139222 learned_addr learned my addr 172.21.15.47:0/2002139222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.330+0000 7fbde08c9700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbddc103440 0x7fbddc19b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde18cb700 1 -- 172.21.15.47:0/2002139222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbddc103440 msgr2=0x7fbddc19b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde18cb700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbddc103440 0x7fbddc19b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde18cb700 1 -- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 msgr2=0x7fbddc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde18cb700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde18cb700 1 -- 172.21.15.47:0/2002139222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbddc1a60a0 con 0x7fbddc1077e0 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde08c9700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbddc103440 0x7fbddc19b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde10ca700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde18cb700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc1077e0 0x7fbddc1a5b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400efb0 tx=0x7fbdd400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbdd27fc700 1 -- 172.21.15.47:0/2002139222 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4003a00 con 0x7fbddc1077e0 2024-05-11T12:21:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbdd27fc700 1 -- 172.21.15.47:0/2002139222 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd400ce50 con 0x7fbddc1077e0 2024-05-11T12:21:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbdd27fc700 1 -- 172.21.15.47:0/2002139222 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4004710 con 0x7fbddc1077e0 2024-05-11T12:21:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbddc1a62d0 con 0x7fbddc1077e0 2024-05-11T12:21:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.331+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbddc1a6790 con 0x7fbddc1077e0 2024-05-11T12:21:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.332+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbddc103b90 con 0x7fbddc1077e0 2024-05-11T12:21:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.333+0000 7fbdd27fc700 1 -- 172.21.15.47:0/2002139222 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbdd4012050 con 0x7fbddc1077e0 2024-05-11T12:21:00.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.334+0000 7fbdd27fc700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbdc80b2900 0x7fbdc80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:00.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.334+0000 7fbde10ca700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbdc80b2900 0x7fbdc80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:00.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.334+0000 7fbdd27fc700 1 -- 172.21.15.47:0/2002139222 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fbdd40e15f0 con 0x7fbddc1077e0 2024-05-11T12:21:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.335+0000 7fbde10ca700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbdc80b2900 0x7fbdc80b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009990 tx=0x7fbdd8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.339+0000 7fbdd27fc700 1 -- 172.21.15.47:0/2002139222 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbdd4022060 con 0x7fbddc1077e0 2024-05-11T12:21:00.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.635+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbddc02e730 con 0x7fbddc1077e0 2024-05-11T12:21:00.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.635+0000 7fbdd27fc700 1 -- 172.21.15.47:0/2002139222 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7fbdd4010040 con 0x7fbddc1077e0 2024-05-11T12:21:00.635 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:00.636 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-11T12:21:00.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbdc80b2900 msgr2=0x7fbdc80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:00.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbdc80b2900 0x7fbdc80b4dc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009990 tx=0x7fbdd8005960 comp rx=0 tx=0).stop 2024-05-11T12:21:00.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc1077e0 msgr2=0x7fbddc1a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc1077e0 0x7fbddc1a5b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400efb0 tx=0x7fbdd400c3a0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 shutdown_connections 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbddc1077e0 0x7fbddc1a5b60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbddc102a90 0x7fbddc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbdc80b2900 0x7fbdc80b4dc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 --2- 172.21.15.47:0/2002139222 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbddc103440 0x7fbddc19b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 >> 172.21.15.47:0/2002139222 conn(0x7fbddc079400 msgr2=0x7fbddc100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:00.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 shutdown_connections 2024-05-11T12:21:00.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:00.638+0000 7fbde332e700 1 -- 172.21.15.47:0/2002139222 wait complete. 2024-05-11T12:21:00.640 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 13 2024-05-11T12:21:00.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:00 smithi064 ceph-mon[34537]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:00.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:00.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:00.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:00.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:00 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:00.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:00 smithi050 ceph-mon[34072]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:00 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:00 smithi047 ceph-mon[30375]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:00 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:01.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-11T12:21:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:01 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2002139222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:21:01.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:01.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:01 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2002139222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:21:01.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:01.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:01 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2002139222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:21:01.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:02.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:02 smithi064 ceph-mon[34537]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:02.911 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:02 smithi047 ceph-mon[30375]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:02.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:02 smithi050 ceph-mon[34072]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 -- 172.21.15.47:0/2756279902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68108900 msgr2=0x7fba68110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 --2- 172.21.15.47:0/2756279902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68108900 0x7fba68110480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fba6000b0d0 tx=0x7fba6000b3e0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 -- 172.21.15.47:0/2756279902 shutdown_connections 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 --2- 172.21.15.47:0/2756279902 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68108900 0x7fba68110480 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 --2- 172.21.15.47:0/2756279902 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba68103800 0x7fba68103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 --2- 172.21.15.47:0/2756279902 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba681041b0 0x7fba681081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 -- 172.21.15.47:0/2756279902 >> 172.21.15.47:0/2756279902 conn(0x7fba6807a6b0 msgr2=0x7fba6807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:03.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 -- 172.21.15.47:0/2756279902 shutdown_connections 2024-05-11T12:21:03.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.469+0000 7fba703dd700 1 -- 172.21.15.47:0/2756279902 wait complete. 2024-05-11T12:21:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 Processor -- start 2024-05-11T12:21:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 -- start start 2024-05-11T12:21:03.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68103800 0x7fba68075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba681041b0 0x7fba68075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba68108900 0x7fba680762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fba681136f0 con 0x7fba68103800 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fba68113570 con 0x7fba68108900 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba703dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fba68113870 con 0x7fba681041b0 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba6e179700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68103800 0x7fba68075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba6e179700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68103800 0x7fba68075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56854/0 (socket says 172.21.15.47:56854) 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba6e179700 1 -- 172.21.15.47:0/1138204280 learned_addr learned my addr 172.21.15.47:0/1138204280 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba6d978700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba681041b0 0x7fba68075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.470+0000 7fba6e97a700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba68108900 0x7fba680762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba6e179700 1 -- 172.21.15.47:0/1138204280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba681041b0 msgr2=0x7fba68075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba6e179700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba681041b0 0x7fba68075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba6e179700 1 -- 172.21.15.47:0/1138204280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba68108900 msgr2=0x7fba680762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba6e179700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba68108900 0x7fba680762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba6e179700 1 -- 172.21.15.47:0/1138204280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba681a80b0 con 0x7fba68103800 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba6e97a700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba68108900 0x7fba680762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba6e179700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68103800 0x7fba68075850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c970 tx=0x7fba6400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba5f7fe700 1 -- 172.21.15.47:0/1138204280 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64007aa0 con 0x7fba68103800 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba681a83a0 con 0x7fba68103800 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba5f7fe700 1 -- 172.21.15.47:0/1138204280 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba64004ce0 con 0x7fba68103800 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba5f7fe700 1 -- 172.21.15.47:0/1138204280 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64005510 con 0x7fba68103800 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.471+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba681a88e0 con 0x7fba68103800 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.472+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba6810bad0 con 0x7fba68103800 2024-05-11T12:21:03.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.473+0000 7fba5f7fe700 1 -- 172.21.15.47:0/1138204280 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fba64007c40 con 0x7fba68103800 2024-05-11T12:21:03.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.474+0000 7fba5f7fe700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fba540b2880 0x7fba540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:03.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.474+0000 7fba6d978700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fba540b2880 0x7fba540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:03.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.474+0000 7fba5f7fe700 1 -- 172.21.15.47:0/1138204280 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fba640e1e80 con 0x7fba68103800 2024-05-11T12:21:03.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.474+0000 7fba6d978700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fba540b2880 0x7fba540b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba68103160 tx=0x7fba580073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:03.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.477+0000 7fba5f7fe700 1 -- 172.21.15.47:0/1138204280 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba640aa0c0 con 0x7fba68103800 2024-05-11T12:21:03.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.723+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fba6810b4b0 con 0x7fba68103800 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.724+0000 7fba5f7fe700 1 -- 172.21.15.47:0/1138204280 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fba640a99c0 con 0x7fba68103800 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:21:03.725 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-05-11T12:21:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.726+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fba540b2880 msgr2=0x7fba540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.726+0000 7fba703dd700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fba540b2880 0x7fba540b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba68103160 tx=0x7fba580073d0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.726+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68103800 msgr2=0x7fba68075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:03.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.726+0000 7fba703dd700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68103800 0x7fba68075850 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fba6400c970 tx=0x7fba6400cc80 comp rx=0 tx=0).stop 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 shutdown_connections 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fba540b2880 0x7fba540b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fba68108900 0x7fba680762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba68103800 0x7fba68075850 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 --2- 172.21.15.47:0/1138204280 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fba681041b0 0x7fba68075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 >> 172.21.15.47:0/1138204280 conn(0x7fba6807a6b0 msgr2=0x7fba68101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 shutdown_connections 2024-05-11T12:21:03.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:03.727+0000 7fba703dd700 1 -- 172.21.15.47:0/1138204280 wait complete. 2024-05-11T12:21:04.343 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-11T12:21:04.343 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-11T12:21:04.606 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:04.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:04.731 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1138204280' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-11T12:21:04.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:21:04.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:21:04.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:04 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:21:04.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:04.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1138204280' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-11T12:21:04.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:21:04.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:21:04.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:04 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:21:04.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:04.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1138204280' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-11T12:21:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:21:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:21:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:04.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:04 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:21:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 -- 172.21.15.47:0/511735924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06b230 msgr2=0x7f6f4c06b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:06.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/511735924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06b230 0x7f6f4c06b6b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6f400098f0 tx=0x7f6f40009c00 comp rx=0 tx=0).stop 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 -- 172.21.15.47:0/511735924 shutdown_connections 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/511735924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c06bbf0 0x7f6f4c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/511735924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c1095c0 0x7f6f4c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/511735924 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06b230 0x7f6f4c06b6b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 -- 172.21.15.47:0/511735924 >> 172.21.15.47:0/511735924 conn(0x7f6f4c079400 msgr2=0x7f6f4c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 -- 172.21.15.47:0/511735924 shutdown_connections 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.345+0000 7f6f50e2f700 1 -- 172.21.15.47:0/511735924 wait complete. 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 Processor -- start 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 -- start start 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c06b230 0x7f6f4c19f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06bbf0 0x7f6f4c19fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c1095c0 0x7f6f4c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:06.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c113720 con 0x7f6f4c1095c0 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1135a0 con 0x7f6f4c06b230 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.346+0000 7f6f50e2f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c1138a0 con 0x7f6f4c06bbf0 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c1095c0 0x7f6f4c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f49d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06bbf0 0x7f6f4c19fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c1095c0 0x7f6f4c199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56864/0 (socket says 172.21.15.47:56864) 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4ad9d700 1 -- 172.21.15.47:0/2478880160 learned_addr learned my addr 172.21.15.47:0/2478880160 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4a59c700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c06b230 0x7f6f4c19f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4a59c700 1 -- 172.21.15.47:0/2478880160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06bbf0 msgr2=0x7f6f4c19fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4a59c700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06bbf0 0x7f6f4c19fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4a59c700 1 -- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c1095c0 msgr2=0x7f6f4c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4a59c700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c1095c0 0x7f6f4c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:06.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4a59c700 1 -- 172.21.15.47:0/2478880160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c19a240 con 0x7f6f4c06b230 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f49d9b700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06bbf0 0x7f6f4c19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4ad9d700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c1095c0 0x7f6f4c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f4a59c700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c06b230 0x7f6f4c19f920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9f0 tx=0x7f6f3c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f3b7fe700 1 -- 172.21.15.47:0/2478880160 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00cb70 con 0x7f6f4c06b230 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c19a530 con 0x7f6f4c06b230 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f3b7fe700 1 -- 172.21.15.47:0/2478880160 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f3c00ee90 con 0x7f6f4c06b230 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f3b7fe700 1 -- 172.21.15.47:0/2478880160 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c003d40 con 0x7f6f4c06b230 2024-05-11T12:21:06.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.347+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c1a67f0 con 0x7f6f4c06b230 2024-05-11T12:21:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.348+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f4c06a240 con 0x7f6f4c06b230 2024-05-11T12:21:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.349+0000 7f6f3b7fe700 1 -- 172.21.15.47:0/2478880160 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6f3c004020 con 0x7f6f4c06b230 2024-05-11T12:21:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.350+0000 7f6f3b7fe700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f340b2900 0x7f6f340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.350+0000 7f6f49d9b700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f340b2900 0x7f6f340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.350+0000 7f6f3b7fe700 1 -- 172.21.15.47:0/2478880160 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f6f3c0e1d80 con 0x7f6f4c06b230 2024-05-11T12:21:06.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.351+0000 7f6f49d9b700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f340b2900 0x7f6f340b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6f400098c0 tx=0x7f6f40019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:06.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.355+0000 7f6f3b7fe700 1 -- 172.21.15.47:0/2478880160 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f3c0e5030 con 0x7f6f4c06b230 2024-05-11T12:21:06.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:06.570+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6f4c02e570 con 0x7f6f4c06b230 2024-05-11T12:21:06.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:06 smithi064 ceph-mon[34537]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:06.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:06 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:06 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2478880160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T12:21:06.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:06 smithi064 ceph-mon[34537]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T12:21:06.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:06 smithi050 ceph-mon[34072]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:06 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:06 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2478880160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T12:21:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:06 smithi050 ceph-mon[34072]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T12:21:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:06 smithi047 ceph-mon[30375]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:06 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:06 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2478880160' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T12:21:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:06 smithi047 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-11T12:21:07.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.496+0000 7f6f3b7fe700 1 -- 172.21.15.47:0/2478880160 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f6f3c0a9f20 con 0x7f6f4c06b230 2024-05-11T12:21:07.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f340b2900 msgr2=0x7f6f340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f340b2900 0x7f6f340b4dc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6f400098c0 tx=0x7f6f40019040 comp rx=0 tx=0).stop 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c06b230 msgr2=0x7f6f4c19f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c06b230 0x7f6f4c19f920 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9f0 tx=0x7f6f3c00ed00 comp rx=0 tx=0).stop 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 shutdown_connections 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f4c1095c0 0x7f6f4c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f340b2900 0x7f6f340b4dc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f4c06b230 0x7f6f4c19f920 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 --2- 172.21.15.47:0/2478880160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f4c06bbf0 0x7f6f4c19fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 >> 172.21.15.47:0/2478880160 conn(0x7f6f4c079400 msgr2=0x7f6f4c1012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 shutdown_connections 2024-05-11T12:21:07.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:07.501+0000 7f6f50e2f700 1 -- 172.21.15.47:0/2478880160 wait complete. 2024-05-11T12:21:07.503 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-05-11T12:21:08.200 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-11T12:21:08.456 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: osdmap e73: 12 total, 12 up, 12 in 2024-05-11T12:21:08.717 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:08 smithi047 ceph-mon[30375]: pgmap v286: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: osdmap e73: 12 total, 12 up, 12 in 2024-05-11T12:21:08.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:08 smithi064 ceph-mon[34537]: pgmap v286: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: osdmap e73: 12 total, 12 up, 12 in 2024-05-11T12:21:08.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:08 smithi050 ceph-mon[34072]: pgmap v286: 33 pgs: 5 creating+peering, 1 active+clean, 27 unknown; 577 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:09.670 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:09 smithi047 ceph-mon[30375]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T12:21:09.671 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:09 smithi047 ceph-mon[30375]: osdmap e74: 12 total, 12 up, 12 in 2024-05-11T12:21:09.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:09 smithi064 ceph-mon[34537]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T12:21:09.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:09 smithi064 ceph-mon[34537]: osdmap e74: 12 total, 12 up, 12 in 2024-05-11T12:21:09.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:09 smithi050 ceph-mon[34072]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-11T12:21:09.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:09 smithi050 ceph-mon[34072]: osdmap e74: 12 total, 12 up, 12 in 2024-05-11T12:21:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 -- 172.21.15.47:0/507513372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a01095b0 msgr2=0x7f12a0109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 --2- 172.21.15.47:0/507513372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a01095b0 0x7f12a0109990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1288009990 tx=0x7f1288009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 -- 172.21.15.47:0/507513372 shutdown_connections 2024-05-11T12:21:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 --2- 172.21.15.47:0/507513372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12a006c480 0x7f12a01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 --2- 172.21.15.47:0/507513372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a01095b0 0x7f12a0109990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 --2- 172.21.15.47:0/507513372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a006bac0 0x7f12a006bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 -- 172.21.15.47:0/507513372 >> 172.21.15.47:0/507513372 conn(0x7f12a00793c0 msgr2=0x7f12a00797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 -- 172.21.15.47:0/507513372 shutdown_connections 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.213+0000 7f12a721b700 1 -- 172.21.15.47:0/507513372 wait complete. 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 Processor -- start 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 -- start start 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a006bac0 0x7f12a0102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a006c480 0x7f12a0102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12a01095b0 0x7f12a0102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f12a01136e0 con 0x7f12a006bac0 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0113560 con 0x7f12a01095b0 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a721b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f12a0113860 con 0x7f12a006c480 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.214+0000 7f12a4fb7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a006bac0 0x7f12a0102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:10.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a006bac0 0x7f12a0102510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36904/0 (socket says 172.21.15.47:36904) 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 -- 172.21.15.47:0/2984730074 learned_addr learned my addr 172.21.15.47:0/2984730074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f129ffff700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a006c480 0x7f12a0102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 -- 172.21.15.47:0/2984730074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a006c480 msgr2=0x7f12a0102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a006c480 0x7f12a0102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 -- 172.21.15.47:0/2984730074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12a01095b0 msgr2=0x7f12a0102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12a01095b0 0x7f12a0102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 -- 172.21.15.47:0/2984730074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a01a81c0 con 0x7f12a006bac0 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f129ffff700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a006c480 0x7f12a0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a4fb7700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a006bac0 0x7f12a0102510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f12880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f129dffb700 1 -- 172.21.15.47:0/2984730074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288027070 con 0x7f12a006bac0 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f129dffb700 1 -- 172.21.15.47:0/2984730074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12880041d0 con 0x7f12a006bac0 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a01a8450 con 0x7f12a006bac0 2024-05-11T12:21:10.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f129dffb700 1 -- 172.21.15.47:0/2984730074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128800f460 con 0x7f12a006bac0 2024-05-11T12:21:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.215+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a01a8900 con 0x7f12a006bac0 2024-05-11T12:21:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.216+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a010ba90 con 0x7f12a006bac0 2024-05-11T12:21:10.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.218+0000 7f129dffb700 1 -- 172.21.15.47:0/2984730074 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f128802a070 con 0x7f12a006bac0 2024-05-11T12:21:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.219+0000 7f129dffb700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f128c0b2900 0x7f128c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.219+0000 7f129ffff700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f128c0b2900 0x7f128c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.219+0000 7f129dffb700 1 -- 172.21.15.47:0/2984730074 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f12880e32f0 con 0x7f12a006bac0 2024-05-11T12:21:10.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.219+0000 7f129ffff700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f128c0b2900 0x7f128c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f12a0079a30 tx=0x7f12900076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:10.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.222+0000 7f129dffb700 1 -- 172.21.15.47:0/2984730074 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1288031070 con 0x7f12a006bac0 2024-05-11T12:21:10.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.438+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f12a010b470 con 0x7f12a006bac0 2024-05-11T12:21:10.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.504+0000 7f129dffb700 1 -- 172.21.15.47:0/2984730074 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v76) ==== 170+0+0 (secure 0 0 0) 0x7f1288013070 con 0x7f12a006bac0 2024-05-11T12:21:10.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.508+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f128c0b2900 msgr2=0x7f128c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:10.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.508+0000 7f12a721b700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f128c0b2900 0x7f128c0b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f12a0079a30 tx=0x7f12900076a0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.508+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a006bac0 msgr2=0x7f12a0102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:10.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.508+0000 7f12a721b700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a006bac0 0x7f12a0102510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1288000c00 tx=0x7f12880047b0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.509+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 shutdown_connections 2024-05-11T12:21:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.509+0000 7f12a721b700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f128c0b2900 0x7f128c0b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.509+0000 7f12a721b700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f12a01095b0 0x7f12a0102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.509+0000 7f12a721b700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f12a006bac0 0x7f12a0102510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.509+0000 7f12a721b700 1 --2- 172.21.15.47:0/2984730074 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f12a006c480 0x7f12a0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.509+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 >> 172.21.15.47:0/2984730074 conn(0x7f12a00793c0 msgr2=0x7f12a0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.511+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 shutdown_connections 2024-05-11T12:21:10.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:10.512+0000 7f12a721b700 1 -- 172.21.15.47:0/2984730074 wait complete. 2024-05-11T12:21:10.516 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-05-11T12:21:10.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:10 smithi047 ceph-mon[30375]: osdmap e75: 12 total, 12 up, 12 in 2024-05-11T12:21:10.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:10 smithi047 ceph-mon[30375]: pgmap v289: 33 pgs: 5 creating+peering, 4 active+clean, 24 unknown; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:10.532 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:10 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2984730074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-11T12:21:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:10 smithi064 ceph-mon[34537]: osdmap e75: 12 total, 12 up, 12 in 2024-05-11T12:21:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:10 smithi064 ceph-mon[34537]: pgmap v289: 33 pgs: 5 creating+peering, 4 active+clean, 24 unknown; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:10 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2984730074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-11T12:21:10.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:10 smithi050 ceph-mon[34072]: osdmap e75: 12 total, 12 up, 12 in 2024-05-11T12:21:10.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:10 smithi050 ceph-mon[34072]: pgmap v289: 33 pgs: 5 creating+peering, 4 active+clean, 24 unknown; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:10.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:10 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2984730074' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-11T12:21:11.325 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-11T12:21:11.578 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:11 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2984730074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-11T12:21:11.578 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:11 smithi047 ceph-mon[30375]: osdmap e76: 12 total, 12 up, 12 in 2024-05-11T12:21:11.607 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:11 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2984730074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-11T12:21:11.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:11 smithi064 ceph-mon[34537]: osdmap e76: 12 total, 12 up, 12 in 2024-05-11T12:21:11.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:11 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2984730074' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-11T12:21:11.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:11 smithi050 ceph-mon[34072]: osdmap e76: 12 total, 12 up, 12 in 2024-05-11T12:21:12.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:12 smithi047 ceph-mon[30375]: osdmap e77: 12 total, 12 up, 12 in 2024-05-11T12:21:12.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:12 smithi047 ceph-mon[30375]: pgmap v292: 97 pgs: 17 creating+peering, 16 active+clean, 64 unknown; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:12 smithi064 ceph-mon[34537]: osdmap e77: 12 total, 12 up, 12 in 2024-05-11T12:21:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:12 smithi064 ceph-mon[34537]: pgmap v292: 97 pgs: 17 creating+peering, 16 active+clean, 64 unknown; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:12.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:12 smithi050 ceph-mon[34072]: osdmap e77: 12 total, 12 up, 12 in 2024-05-11T12:21:12.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:12 smithi050 ceph-mon[34072]: pgmap v292: 97 pgs: 17 creating+peering, 16 active+clean, 64 unknown; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 -- 172.21.15.47:0/3327461700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d01095f0 msgr2=0x7f14d01099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 --2- 172.21.15.47:0/3327461700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d01095f0 0x7f14d01099d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009990 tx=0x7f14b8009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 -- 172.21.15.47:0/3327461700 shutdown_connections 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 --2- 172.21.15.47:0/3327461700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d006c680 0x7f14d0110430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 --2- 172.21.15.47:0/3327461700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d01095f0 0x7f14d01099d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 --2- 172.21.15.47:0/3327461700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d006bcc0 0x7f14d006c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 -- 172.21.15.47:0/3327461700 >> 172.21.15.47:0/3327461700 conn(0x7f14d00784c0 msgr2=0x7f14d00788d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 -- 172.21.15.47:0/3327461700 shutdown_connections 2024-05-11T12:21:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 -- 172.21.15.47:0/3327461700 wait complete. 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 Processor -- start 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.322+0000 7f14d6c22700 1 -- start start 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d6c22700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d006bcc0 0x7f14d019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d6c22700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d006c680 0x7f14d019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d6c22700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d01095f0 0x7f14d0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d6c22700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0113730 con 0x7f14d006c680 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d6c22700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01135b0 con 0x7f14d006bcc0 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d6c22700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f14d01138b0 con 0x7f14d01095f0 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d006c680 0x7f14d019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d006c680 0x7f14d019fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36920/0 (socket says 172.21.15.47:36920) 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 -- 172.21.15.47:0/1402987646 learned_addr learned my addr 172.21.15.47:0/1402987646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d51bf700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d01095f0 0x7f14d0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d49be700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d006bcc0 0x7f14d019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:13.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 -- 172.21.15.47:0/1402987646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d01095f0 msgr2=0x7f14d0199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d01095f0 0x7f14d0199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 -- 172.21.15.47:0/1402987646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d006bcc0 msgr2=0x7f14d019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d006bcc0 0x7f14d019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 -- 172.21.15.47:0/1402987646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d019a190 con 0x7f14d006c680 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d49be700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d006bcc0 0x7f14d019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14d51bf700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d01095f0 0x7f14d0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.323+0000 7f14cffff700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d006c680 0x7f14d019fdb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14c0007f10 tx=0x7f14c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.324+0000 7f14cdffb700 1 -- 172.21.15.47:0/1402987646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c000ddd0 con 0x7f14d006c680 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.324+0000 7f14cdffb700 1 -- 172.21.15.47:0/1402987646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c0004bb0 con 0x7f14d006c680 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.324+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d019a480 con 0x7f14d006c680 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.324+0000 7f14cdffb700 1 -- 172.21.15.47:0/1402987646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00046a0 con 0x7f14d006c680 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.324+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01a6740 con 0x7f14d006c680 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.325+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d010baf0 con 0x7f14d006c680 2024-05-11T12:21:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.326+0000 7f14cdffb700 1 -- 172.21.15.47:0/1402987646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f14c001c020 con 0x7f14d006c680 2024-05-11T12:21:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.327+0000 7f14cdffb700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f14bc0b2900 0x7f14bc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.327+0000 7f14d49be700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f14bc0b2900 0x7f14bc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.327+0000 7f14cdffb700 1 -- 172.21.15.47:0/1402987646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f14c00e1940 con 0x7f14d006c680 2024-05-11T12:21:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.327+0000 7f14d49be700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f14bc0b2900 0x7f14bc0b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14b8006010 tx=0x7f14b800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:13.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.332+0000 7f14cdffb700 1 -- 172.21.15.47:0/1402987646 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f14c0051df0 con 0x7f14d006c680 2024-05-11T12:21:13.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:13.601+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f14d010b4d0 con 0x7f14d006c680 2024-05-11T12:21:13.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:13 smithi064 ceph-mon[34537]: osdmap e78: 12 total, 12 up, 12 in 2024-05-11T12:21:13.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:13 smithi050 ceph-mon[34072]: osdmap e78: 12 total, 12 up, 12 in 2024-05-11T12:21:14.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:13 smithi047 ceph-mon[30375]: osdmap e78: 12 total, 12 up, 12 in 2024-05-11T12:21:14.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.589+0000 7f14cdffb700 1 -- 172.21.15.47:0/1402987646 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f14c0010070 con 0x7f14d006c680 2024-05-11T12:21:14.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.593+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f14bc0b2900 msgr2=0x7f14bc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.593+0000 7f14d6c22700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f14bc0b2900 0x7f14bc0b4dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f14b8006010 tx=0x7f14b800b540 comp rx=0 tx=0).stop 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.593+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d006c680 msgr2=0x7f14d019fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.593+0000 7f14d6c22700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d006c680 0x7f14d019fdb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14c0007f10 tx=0x7f14c000d3b0 comp rx=0 tx=0).stop 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 shutdown_connections 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f14d01095f0 0x7f14d0199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f14bc0b2900 0x7f14bc0b4dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14d006bcc0 0x7f14d019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 --2- 172.21.15.47:0/1402987646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f14d006c680 0x7f14d019fdb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 >> 172.21.15.47:0/1402987646 conn(0x7f14d00784c0 msgr2=0x7f14d01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 shutdown_connections 2024-05-11T12:21:14.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:14.594+0000 7f14d6c22700 1 -- 172.21.15.47:0/1402987646 wait complete. 2024-05-11T12:21:14.596 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-05-11T12:21:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:14 smithi064 ceph-mon[34537]: pgmap v294: 97 pgs: 12 creating+peering, 73 active+clean, 12 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:14 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1402987646' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-11T12:21:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:14 smithi064 ceph-mon[34537]: daemon mds.h assigned to filesystem cephfs as rank 0 2024-05-11T12:21:14.913 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:14 smithi047 ceph-mon[30375]: pgmap v294: 97 pgs: 12 creating+peering, 73 active+clean, 12 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:14.913 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:14 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1402987646' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-11T12:21:14.913 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:14 smithi047 ceph-mon[30375]: daemon mds.h assigned to filesystem cephfs as rank 0 2024-05-11T12:21:14.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:14 smithi050 ceph-mon[34072]: pgmap v294: 97 pgs: 12 creating+peering, 73 active+clean, 12 unknown; 577 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:21:14.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:14 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1402987646' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-11T12:21:14.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:14 smithi050 ceph-mon[34072]: daemon mds.h assigned to filesystem cephfs as rank 0 2024-05-11T12:21:15.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:21:15.579 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:15.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:15 smithi064 ceph-mon[34537]: osdmap e79: 12 total, 12 up, 12 in 2024-05-11T12:21:15.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:15 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1402987646' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-11T12:21:15.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:15 smithi064 ceph-mon[34537]: fsmap cephfs:1 {0=h=up:creating} 11 up:standby 2024-05-11T12:21:15.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:15 smithi064 ceph-mon[34537]: daemon mds.h is now active in filesystem cephfs as rank 0 2024-05-11T12:21:15.909 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:15 smithi047 ceph-mon[30375]: osdmap e79: 12 total, 12 up, 12 in 2024-05-11T12:21:15.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:15 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1402987646' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-11T12:21:15.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:15 smithi047 ceph-mon[30375]: fsmap cephfs:1 {0=h=up:creating} 11 up:standby 2024-05-11T12:21:15.910 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:15 smithi047 ceph-mon[30375]: daemon mds.h is now active in filesystem cephfs as rank 0 2024-05-11T12:21:15.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:15 smithi050 ceph-mon[34072]: osdmap e79: 12 total, 12 up, 12 in 2024-05-11T12:21:15.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:15 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1402987646' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-11T12:21:15.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:15 smithi050 ceph-mon[34072]: fsmap cephfs:1 {0=h=up:creating} 11 up:standby 2024-05-11T12:21:15.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:15 smithi050 ceph-mon[34072]: daemon mds.h is now active in filesystem cephfs as rank 0 2024-05-11T12:21:16.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:16 smithi047 ceph-mon[30375]: pgmap v296: 97 pgs: 12 creating+peering, 85 active+clean; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 611 B/s wr, 1 op/s 2024-05-11T12:21:16.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:16 smithi047 ceph-mon[30375]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-11T12:21:16.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:16 smithi047 ceph-mon[30375]: Cluster is now healthy 2024-05-11T12:21:16.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:16 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] up:active 2024-05-11T12:21:16.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:16 smithi047 ceph-mon[30375]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:16.698 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:16 smithi047 ceph-mon[30375]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:16 smithi064 ceph-mon[34537]: pgmap v296: 97 pgs: 12 creating+peering, 85 active+clean; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 611 B/s wr, 1 op/s 2024-05-11T12:21:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:16 smithi064 ceph-mon[34537]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-11T12:21:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:16 smithi064 ceph-mon[34537]: Cluster is now healthy 2024-05-11T12:21:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:16 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] up:active 2024-05-11T12:21:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:16 smithi064 ceph-mon[34537]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:16.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:16 smithi064 ceph-mon[34537]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:16.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:16 smithi050 ceph-mon[34072]: pgmap v296: 97 pgs: 12 creating+peering, 85 active+clean; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 611 B/s wr, 1 op/s 2024-05-11T12:21:16.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:16 smithi050 ceph-mon[34072]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-11T12:21:16.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:16 smithi050 ceph-mon[34072]: Cluster is now healthy 2024-05-11T12:21:16.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:16 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] up:active 2024-05-11T12:21:16.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:16 smithi050 ceph-mon[34072]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:16.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:16 smithi050 ceph-mon[34072]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 -- 172.21.15.47:0/1387656608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 msgr2=0x7f28e8110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 --2- 172.21.15.47:0/1387656608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8110480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b0d0 tx=0x7f28e000b3e0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 -- 172.21.15.47:0/1387656608 shutdown_connections 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 --2- 172.21.15.47:0/1387656608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8110480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 --2- 172.21.15.47:0/1387656608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28e8103800 0x7f28e8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 --2- 172.21.15.47:0/1387656608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e81041b0 0x7f28e81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 -- 172.21.15.47:0/1387656608 >> 172.21.15.47:0/1387656608 conn(0x7f28e807a6b0 msgr2=0x7f28e807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 -- 172.21.15.47:0/1387656608 shutdown_connections 2024-05-11T12:21:17.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.257+0000 7f28f0561700 1 -- 172.21.15.47:0/1387656608 wait complete. 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 Processor -- start 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 -- start start 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e8103800 0x7f28e819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28e81041b0 0x7f28e819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f28e81136d0 con 0x7f28e8108900 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8113550 con 0x7f28e81041b0 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28f0561700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f28e8113850 con 0x7f28e8103800 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28eeafe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28eeafe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48082/0 (socket says 172.21.15.47:48082) 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.258+0000 7f28eeafe700 1 -- 172.21.15.47:0/203301563 learned_addr learned my addr 172.21.15.47:0/203301563 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28ee2fd700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e8103800 0x7f28e819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28edafc700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28e81041b0 0x7f28e819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:17.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28eeafe700 1 -- 172.21.15.47:0/203301563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e8103800 msgr2=0x7f28e819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28eeafe700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e8103800 0x7f28e819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28eeafe700 1 -- 172.21.15.47:0/203301563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28e81041b0 msgr2=0x7f28e819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28eeafe700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28e81041b0 0x7f28e819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28eeafe700 1 -- 172.21.15.47:0/203301563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e819a090 con 0x7f28e8108900 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28ee2fd700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e8103800 0x7f28e819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28edafc700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28e81041b0 0x7f28e819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28eeafe700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8199930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f28e00095c0 tx=0x7f28e0007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28df7fe700 1 -- 172.21.15.47:0/203301563 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e001f070 con 0x7f28e8108900 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28df7fe700 1 -- 172.21.15.47:0/203301563 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28e0003c00 con 0x7f28e8108900 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e819a320 con 0x7f28e8108900 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28df7fe700 1 -- 172.21.15.47:0/203301563 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28e001b6a0 con 0x7f28e8108900 2024-05-11T12:21:17.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.259+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e81a65f0 con 0x7f28e8108900 2024-05-11T12:21:17.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.260+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e806a240 con 0x7f28e8108900 2024-05-11T12:21:17.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.262+0000 7f28df7fe700 1 -- 172.21.15.47:0/203301563 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f28e0022070 con 0x7f28e8108900 2024-05-11T12:21:17.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.262+0000 7f28df7fe700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f28d40b2900 0x7f28d40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:17.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.262+0000 7f28df7fe700 1 -- 172.21.15.47:0/203301563 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f28e000f020 con 0x7f28e8108900 2024-05-11T12:21:17.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.262+0000 7f28ee2fd700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f28d40b2900 0x7f28d40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:17.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.263+0000 7f28ee2fd700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f28d40b2900 0x7f28d40b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28e4005950 tx=0x7f28e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:17.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.266+0000 7f28df7fe700 1 -- 172.21.15.47:0/203301563 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28e0029020 con 0x7f28e8108900 2024-05-11T12:21:17.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.485+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28e819ae10 con 0x7f28e8108900 2024-05-11T12:21:17.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.487+0000 7f28df7fe700 1 -- 172.21.15.47:0/203301563 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22111 (secure 0 0 0) 0x7f28e00ab6e0 con 0x7f28e8108900 2024-05-11T12:21:17.488 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:17.488 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:21:13.603487+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:21:17.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f28d40b2900 msgr2=0x7f28d40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:17.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f28d40b2900 0x7f28d40b4dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f28e4005950 tx=0x7f28e40058e0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 msgr2=0x7f28e8199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8199930 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f28e00095c0 tx=0x7f28e0007ad0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 shutdown_connections 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28e8108900 0x7f28e8199930 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f28e8103800 0x7f28e819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f28d40b2900 0x7f28d40b4dc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 --2- 172.21.15.47:0/203301563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f28e81041b0 0x7f28e819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.489+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 >> 172.21.15.47:0/203301563 conn(0x7f28e807a6b0 msgr2=0x7f28e81015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.490+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 shutdown_connections 2024-05-11T12:21:17.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:17.490+0000 7f28f0561700 1 -- 172.21.15.47:0/203301563 wait complete. 2024-05-11T12:21:17.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:17 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/203301563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:17.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:17 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/203301563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:17.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:17 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/203301563' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:18.102 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:21:18.355 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:18.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:18 smithi047 ceph-mon[30375]: pgmap v297: 97 pgs: 97 active+clean; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-11T12:21:18.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:18 smithi064 ceph-mon[34537]: pgmap v297: 97 pgs: 97 active+clean; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-11T12:21:18.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:18 smithi050 ceph-mon[34072]: pgmap v297: 97 pgs: 97 active+clean; 578 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-05-11T12:21:18.949 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:21:18 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h[64152]: 2024-05-11T12:21:18.772+0000 7f961dfab700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-11T12:21:20.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.092+0000 7f160842b700 1 -- 172.21.15.47:0/2628917279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600103800 msgr2=0x7f1600103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:20.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.092+0000 7f160842b700 1 --2- 172.21.15.47:0/2628917279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600103800 0x7f1600103be0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0099c0 tx=0x7f15fc009cd0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 -- 172.21.15.47:0/2628917279 shutdown_connections 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 --2- 172.21.15.47:0/2628917279 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1600108900 0x7f1600110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 --2- 172.21.15.47:0/2628917279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600103800 0x7f1600103be0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 --2- 172.21.15.47:0/2628917279 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001041b0 0x7f16001081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 -- 172.21.15.47:0/2628917279 >> 172.21.15.47:0/2628917279 conn(0x7f160007a6b0 msgr2=0x7f160007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 -- 172.21.15.47:0/2628917279 shutdown_connections 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 -- 172.21.15.47:0/2628917279 wait complete. 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 Processor -- start 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.093+0000 7f160842b700 1 -- start start 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f160842b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1600103800 0x7f160019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f160842b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001041b0 0x7f160019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f160842b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600108900 0x7f1600199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f160842b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f16001136e0 con 0x7f1600108900 2024-05-11T12:21:20.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f160842b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1600113560 con 0x7f1600103800 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f160842b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1600113860 con 0x7f16001041b0 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600108900 0x7f1600199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600108900 0x7f1600199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48098/0 (socket says 172.21.15.47:48098) 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 -- 172.21.15.47:0/2387302633 learned_addr learned my addr 172.21.15.47:0/2387302633 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16059c6700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001041b0 0x7f160019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16061c7700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1600103800 0x7f160019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 -- 172.21.15.47:0/2387302633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001041b0 msgr2=0x7f160019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001041b0 0x7f160019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 -- 172.21.15.47:0/2387302633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1600103800 msgr2=0x7f160019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1600103800 0x7f160019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 -- 172.21.15.47:0/2387302633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160019a1c0 con 0x7f1600108900 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16059c6700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001041b0 0x7f160019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:21:20.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.094+0000 7f16069c8700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600108900 0x7f1600199a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15f8007ee0 tx=0x7f15f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:20.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.095+0000 7f15f77fe700 1 -- 172.21.15.47:0/2387302633 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f800dbd0 con 0x7f1600108900 2024-05-11T12:21:20.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.095+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160019a4b0 con 0x7f1600108900 2024-05-11T12:21:20.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.095+0000 7f15f77fe700 1 -- 172.21.15.47:0/2387302633 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f800dd70 con 0x7f1600108900 2024-05-11T12:21:20.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.095+0000 7f15f77fe700 1 -- 172.21.15.47:0/2387302633 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f8005990 con 0x7f1600108900 2024-05-11T12:21:20.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.095+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16001a6770 con 0x7f1600108900 2024-05-11T12:21:20.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.096+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160010bb40 con 0x7f1600108900 2024-05-11T12:21:20.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.097+0000 7f15f77fe700 1 -- 172.21.15.47:0/2387302633 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f15f801a020 con 0x7f1600108900 2024-05-11T12:21:20.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.098+0000 7f15f77fe700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15ec0b2880 0x7f15ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:20.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.098+0000 7f16061c7700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15ec0b2880 0x7f15ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:20.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.098+0000 7f15f77fe700 1 -- 172.21.15.47:0/2387302633 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f15f80e17f0 con 0x7f1600108900 2024-05-11T12:21:20.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.098+0000 7f16061c7700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15ec0b2880 0x7f15ec0b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f160019aba0 tx=0x7f15fc005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:20.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.102+0000 7f15f77fe700 1 -- 172.21.15.47:0/2387302633 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15f8010070 con 0x7f1600108900 2024-05-11T12:21:20.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.318+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f160019af90 con 0x7f1600108900 2024-05-11T12:21:20.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.320+0000 7f15f77fe700 1 -- 172.21.15.47:0/2387302633 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+22111 (secure 0 0 0) 0x7f15f80a9660 con 0x7f1600108900 2024-05-11T12:21:20.320 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:20.320 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":79,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:21:13.603487+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15ec0b2880 msgr2=0x7f15ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15ec0b2880 0x7f15ec0b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f160019aba0 tx=0x7f15fc005c40 comp rx=0 tx=0).stop 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600108900 msgr2=0x7f1600199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600108900 0x7f1600199a60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15f8007ee0 tx=0x7f15f800d3b0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 shutdown_connections 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1600108900 0x7f1600199a60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15ec0b2880 0x7f15ec0b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1600103800 0x7f160019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 --2- 172.21.15.47:0/2387302633 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f16001041b0 0x7f160019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 >> 172.21.15.47:0/2387302633 conn(0x7f160007a6b0 msgr2=0x7f1600101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 shutdown_connections 2024-05-11T12:21:20.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:20.322+0000 7f160842b700 1 -- 172.21.15.47:0/2387302633 wait complete. 2024-05-11T12:21:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:20 smithi064 ceph-mon[34537]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-11T12:21:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:20 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2387302633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:20.940 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:20 smithi047 ceph-mon[30375]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-11T12:21:20.940 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:20 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2387302633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:20.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:20 smithi050 ceph-mon[34072]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-11T12:21:20.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:20 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2387302633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:20.971 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-11T12:21:21.226 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:22.732 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:22 smithi047 ceph-mon[30375]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-05-11T12:21:22.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:22 smithi064 ceph-mon[34537]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-05-11T12:21:22.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:22 smithi050 ceph-mon[34072]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 4 op/s 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.982+0000 7fab12767700 1 -- 172.21.15.47:0/277670514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 msgr2=0x7fab0c1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.982+0000 7fab12767700 1 --2- 172.21.15.47:0/277670514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c1103e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab0400b0a0 tx=0x7fab0400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.982+0000 7fab12767700 1 -- 172.21.15.47:0/277670514 shutdown_connections 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.982+0000 7fab12767700 1 --2- 172.21.15.47:0/277670514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c1103e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.982+0000 7fab12767700 1 --2- 172.21.15.47:0/277670514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab0c1095c0 0x7fab0c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.982+0000 7fab12767700 1 --2- 172.21.15.47:0/277670514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab0c06b230 0x7fab0c06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.982+0000 7fab12767700 1 -- 172.21.15.47:0/277670514 >> 172.21.15.47:0/277670514 conn(0x7fab0c079400 msgr2=0x7fab0c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 -- 172.21.15.47:0/277670514 shutdown_connections 2024-05-11T12:21:22.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 -- 172.21.15.47:0/277670514 wait complete. 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 Processor -- start 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 -- start start 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab0c06b230 0x7fab0c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab0c1095c0 0x7fab0c1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.983+0000 7fab12767700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c113720 con 0x7fab0c06bbf0 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab12767700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1135a0 con 0x7fab0c1095c0 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab12767700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c1138a0 con 0x7fab0c06b230 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48112/0 (socket says 172.21.15.47:48112) 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 -- 172.21.15.47:0/3091142012 learned_addr learned my addr 172.21.15.47:0/3091142012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab10d04700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab0c1095c0 0x7fab0c1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab0bfff700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab0c06b230 0x7fab0c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 -- 172.21.15.47:0/3091142012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab0c06b230 msgr2=0x7fab0c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab0c06b230 0x7fab0c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 -- 172.21.15.47:0/3091142012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab0c1095c0 msgr2=0x7fab0c1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab0c1095c0 0x7fab0c1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 -- 172.21.15.47:0/3091142012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c19a140 con 0x7fab0c06bbf0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab10d04700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab0c1095c0 0x7fab0c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab03fff700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c19fd60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faaf400c9b0 tx=0x7faaf400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab09ffb700 1 -- 172.21.15.47:0/3091142012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4007b00 con 0x7fab0c06bbf0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab09ffb700 1 -- 172.21.15.47:0/3091142012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaf4007ca0 con 0x7fab0c06bbf0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c19a430 con 0x7fab0c06bbf0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab09ffb700 1 -- 172.21.15.47:0/3091142012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4018730 con 0x7fab0c06bbf0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.984+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c1a66f0 con 0x7fab0c06bbf0 2024-05-11T12:21:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.985+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab0c06a240 con 0x7fab0c06bbf0 2024-05-11T12:21:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.987+0000 7fab09ffb700 1 -- 172.21.15.47:0/3091142012 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7faaf400f450 con 0x7fab0c06bbf0 2024-05-11T12:21:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.988+0000 7fab09ffb700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faaf80b2900 0x7faaf80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.988+0000 7fab0bfff700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faaf80b2900 0x7faaf80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.988+0000 7fab09ffb700 1 -- 172.21.15.47:0/3091142012 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7faaf40e23d0 con 0x7fab0c06bbf0 2024-05-11T12:21:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.988+0000 7fab0bfff700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faaf80b2900 0x7faaf80b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faafc005fd0 tx=0x7faafc00af00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:22.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:22.992+0000 7fab09ffb700 1 -- 172.21.15.47:0/3091142012 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faaf4014030 con 0x7fab0c06bbf0 2024-05-11T12:21:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.253+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fab0c19af10 con 0x7fab0c06bbf0 2024-05-11T12:21:23.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.603+0000 7fab09ffb700 1 -- 172.21.15.47:0/3091142012 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7faaf40aa350 con 0x7fab0c06bbf0 2024-05-11T12:21:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faaf80b2900 msgr2=0x7faaf80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faaf80b2900 0x7faaf80b4dc0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faafc005fd0 tx=0x7faafc00af00 comp rx=0 tx=0).stop 2024-05-11T12:21:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 msgr2=0x7fab0c19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c19fd60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faaf400c9b0 tx=0x7faaf400cd70 comp rx=0 tx=0).stop 2024-05-11T12:21:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 shutdown_connections 2024-05-11T12:21:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faaf80b2900 0x7faaf80b4dc0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:23.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab0c1095c0 0x7fab0c1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab0c06b230 0x7fab0c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 --2- 172.21.15.47:0/3091142012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab0c06bbf0 0x7fab0c19fd60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 >> 172.21.15.47:0/3091142012 conn(0x7fab0c079400 msgr2=0x7fab0c100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 shutdown_connections 2024-05-11T12:21:23.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:23.606+0000 7fab12767700 1 -- 172.21.15.47:0/3091142012 wait complete. 2024-05-11T12:21:23.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:23 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3091142012' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-11T12:21:23.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:23 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3091142012' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-11T12:21:23.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:23 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3091142012' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-11T12:21:24.376 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-11T12:21:24.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-11T12:21:24.638 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:24.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:24 smithi064 ceph-mon[34537]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-11T12:21:24.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:24 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3091142012' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-11T12:21:24.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:24 smithi064 ceph-mon[34537]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:24.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:24 smithi047 ceph-mon[30375]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-11T12:21:24.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:24 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3091142012' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-11T12:21:24.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:24 smithi047 ceph-mon[30375]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:24.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:24 smithi050 ceph-mon[34072]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-11T12:21:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:24 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3091142012' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-11T12:21:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:24 smithi050 ceph-mon[34072]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 -- 172.21.15.47:0/885005599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006c6e0 msgr2=0x7efef0110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 --2- 172.21.15.47:0/885005599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006c6e0 0x7efef0110410 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efee800b0d0 tx=0x7efee800b3e0 comp rx=0 tx=0).stop 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 -- 172.21.15.47:0/885005599 shutdown_connections 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 --2- 172.21.15.47:0/885005599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006c6e0 0x7efef0110410 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 --2- 172.21.15.47:0/885005599 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efef0109640 0x7efef0109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 --2- 172.21.15.47:0/885005599 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef006bd20 0x7efef006c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 -- 172.21.15.47:0/885005599 >> 172.21.15.47:0/885005599 conn(0x7efef00794f0 msgr2=0x7efef0079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 -- 172.21.15.47:0/885005599 shutdown_connections 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.351+0000 7efef48bd700 1 -- 172.21.15.47:0/885005599 wait complete. 2024-05-11T12:21:26.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.352+0000 7efef48bd700 1 Processor -- start 2024-05-11T12:21:26.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.352+0000 7efef48bd700 1 -- start start 2024-05-11T12:21:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.352+0000 7efef48bd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006bd20 0x7efef01024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.352+0000 7efef48bd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efef006c6e0 0x7efef0102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efef48bd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef0109640 0x7efef0102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efef48bd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efef01137a0 con 0x7efef006bd20 2024-05-11T12:21:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efef48bd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efef0113620 con 0x7efef0109640 2024-05-11T12:21:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efef48bd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efef0113920 con 0x7efef006c6e0 2024-05-11T12:21:26.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006bd20 0x7efef01024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006bd20 0x7efef01024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48142/0 (socket says 172.21.15.47:48142) 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 -- 172.21.15.47:0/45522475 learned_addr learned my addr 172.21.15.47:0/45522475 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeeed9d700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef0109640 0x7efef0102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeedd9b700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efef006c6e0 0x7efef0102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 -- 172.21.15.47:0/45522475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efef006c6e0 msgr2=0x7efef0102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efef006c6e0 0x7efef0102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 -- 172.21.15.47:0/45522475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef0109640 msgr2=0x7efef0102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef0109640 0x7efef0102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 -- 172.21.15.47:0/45522475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef01a8050 con 0x7efef006bd20 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efeee59c700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006bd20 0x7efef01024c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efee000c970 tx=0x7efee000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efedf7fe700 1 -- 172.21.15.47:0/45522475 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0007aa0 con 0x7efef006bd20 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef01a8340 con 0x7efef006bd20 2024-05-11T12:21:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.353+0000 7efedf7fe700 1 -- 172.21.15.47:0/45522475 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee0004ce0 con 0x7efef006bd20 2024-05-11T12:21:26.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.354+0000 7efedf7fe700 1 -- 172.21.15.47:0/45522475 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0005510 con 0x7efef006bd20 2024-05-11T12:21:26.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.354+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01a8880 con 0x7efef006bd20 2024-05-11T12:21:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.355+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef010bad0 con 0x7efef006bd20 2024-05-11T12:21:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.356+0000 7efedf7fe700 1 -- 172.21.15.47:0/45522475 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7efee0007c40 con 0x7efef006bd20 2024-05-11T12:21:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.357+0000 7efedf7fe700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efed80b2900 0x7efed80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.357+0000 7efeedd9b700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efed80b2900 0x7efed80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.357+0000 7efedf7fe700 1 -- 172.21.15.47:0/45522475 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7efee00e1d50 con 0x7efef006bd20 2024-05-11T12:21:26.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.357+0000 7efeedd9b700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efed80b2900 0x7efed80b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efee4005950 tx=0x7efee40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.362+0000 7efedf7fe700 1 -- 172.21.15.47:0/45522475 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efee00e5030 con 0x7efef006bd20 2024-05-11T12:21:26.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:26.623+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7efef010b4b0 con 0x7efef006bd20 2024-05-11T12:21:26.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:26 smithi064 ceph-mon[34537]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-11T12:21:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:26 smithi050 ceph-mon[34072]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-11T12:21:27.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:26 smithi047 ceph-mon[30375]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-11T12:21:27.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.618+0000 7efedf7fe700 1 -- 172.21.15.47:0/45522475 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7efee00a9c50 con 0x7efef006bd20 2024-05-11T12:21:27.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efed80b2900 msgr2=0x7efed80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:27.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efed80b2900 0x7efed80b4dc0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efee4005950 tx=0x7efee40058e0 comp rx=0 tx=0).stop 2024-05-11T12:21:27.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006bd20 msgr2=0x7efef01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:27.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006bd20 0x7efef01024c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7efee000c970 tx=0x7efee000cc80 comp rx=0 tx=0).stop 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 shutdown_connections 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efed80b2900 0x7efed80b4dc0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efef0109640 0x7efef0102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef006bd20 0x7efef01024c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 --2- 172.21.15.47:0/45522475 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efef006c6e0 0x7efef0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.621+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 >> 172.21.15.47:0/45522475 conn(0x7efef00794f0 msgr2=0x7efef0101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.622+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 shutdown_connections 2024-05-11T12:21:27.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:27.622+0000 7efef48bd700 1 -- 172.21.15.47:0/45522475 wait complete. 2024-05-11T12:21:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:27 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/45522475' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-11T12:21:27.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:27 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/45522475' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-11T12:21:27.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:27 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/45522475' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-11T12:21:28.294 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-11T12:21:28.555 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:28.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:28 smithi047 ceph-mon[30375]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-05-11T12:21:28.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:28 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/45522475' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-11T12:21:28.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:28 smithi047 ceph-mon[30375]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:28.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:28 smithi047 ceph-mon[30375]: fsmap cephfs:1 {0=h=up:active} 1 up:standby-replay 10 up:standby 2024-05-11T12:21:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:28 smithi050 ceph-mon[34072]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-05-11T12:21:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:28 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/45522475' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-11T12:21:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:28 smithi050 ceph-mon[34072]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:28 smithi050 ceph-mon[34072]: fsmap cephfs:1 {0=h=up:active} 1 up:standby-replay 10 up:standby 2024-05-11T12:21:29.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:28 smithi064 ceph-mon[34537]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-05-11T12:21:29.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:28 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/45522475' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-11T12:21:29.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:28 smithi064 ceph-mon[34537]: fsmap cephfs:1 {0=h=up:active} 11 up:standby 2024-05-11T12:21:29.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:28 smithi064 ceph-mon[34537]: fsmap cephfs:1 {0=h=up:active} 1 up:standby-replay 10 up:standby 2024-05-11T12:21:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 -- 172.21.15.47:0/1254585449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 msgr2=0x7f48d01088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 --2- 172.21.15.47:0/1254585449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01088a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f48c0009990 tx=0x7f48c0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 -- 172.21.15.47:0/1254585449 shutdown_connections 2024-05-11T12:21:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 --2- 172.21.15.47:0/1254585449 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d0102e40 0x7f48d010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 --2- 172.21.15.47:0/1254585449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01088a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 --2- 172.21.15.47:0/1254585449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48d0102480 0x7f48d0102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 -- 172.21.15.47:0/1254585449 >> 172.21.15.47:0/1254585449 conn(0x7f48d0079400 msgr2=0x7f48d0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 -- 172.21.15.47:0/1254585449 shutdown_connections 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.188+0000 7f48d5281700 1 -- 172.21.15.47:0/1254585449 wait complete. 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.189+0000 7f48d5281700 1 Processor -- start 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.189+0000 7f48d5281700 1 -- start start 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48d5281700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d0102480 0x7f48d019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48d5281700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48d0102e40 0x7f48d019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48d5281700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48d5281700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f48d0112650 con 0x7f48d01084c0 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48d5281700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f48d01124d0 con 0x7f48d0102e40 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48d5281700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f48d01127d0 con 0x7f48d0102480 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44962/0 (socket says 172.21.15.47:44962) 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 -- 172.21.15.47:0/3414447974 learned_addr learned my addr 172.21.15.47:0/3414447974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48ceffd700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d0102480 0x7f48d019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48ce7fc700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48d0102e40 0x7f48d019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 -- 172.21.15.47:0/3414447974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d0102480 msgr2=0x7f48d019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d0102480 0x7f48d019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 -- 172.21.15.47:0/3414447974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48d0102e40 msgr2=0x7f48d019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48d0102e40 0x7f48d019b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 -- 172.21.15.47:0/3414447974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d01a60f0 con 0x7f48d01084c0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.190+0000 7f48cf7fe700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01a5bb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007d70 tx=0x7f48c800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.191+0000 7f48b7fff700 1 -- 172.21.15.47:0/3414447974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8016020 con 0x7f48d01084c0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.191+0000 7f48b7fff700 1 -- 172.21.15.47:0/3414447974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48c800ac90 con 0x7f48d01084c0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.191+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d01a6320 con 0x7f48d01084c0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.191+0000 7f48b7fff700 1 -- 172.21.15.47:0/3414447974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c80115e0 con 0x7f48d01084c0 2024-05-11T12:21:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.191+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d01a6810 con 0x7f48d01084c0 2024-05-11T12:21:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.192+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d006a240 con 0x7f48d01084c0 2024-05-11T12:21:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.193+0000 7f48b7fff700 1 -- 172.21.15.47:0/3414447974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f48c8019020 con 0x7f48d01084c0 2024-05-11T12:21:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.194+0000 7f48b7fff700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f48b80b2880 0x7f48b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.194+0000 7f48ceffd700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f48b80b2880 0x7f48b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.194+0000 7f48b7fff700 1 -- 172.21.15.47:0/3414447974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f48c80e2000 con 0x7f48d01084c0 2024-05-11T12:21:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.195+0000 7f48ceffd700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f48b80b2880 0x7f48b80b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48c00098e0 tx=0x7f48c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:30.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.199+0000 7f48b7fff700 1 -- 172.21.15.47:0/3414447974 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48c8020020 con 0x7f48d01084c0 2024-05-11T12:21:30.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.460+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f48d002e570 con 0x7f48d01084c0 2024-05-11T12:21:30.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.626+0000 7f48b7fff700 1 -- 172.21.15.47:0/3414447974 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f48c80a9e70 con 0x7f48d01084c0 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f48b80b2880 msgr2=0x7f48b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f48b80b2880 0x7f48b80b4d40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f48c00098e0 tx=0x7f48c0005960 comp rx=0 tx=0).stop 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 msgr2=0x7f48d01a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01a5bb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f48c8007d70 tx=0x7f48c800a340 comp rx=0 tx=0).stop 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 shutdown_connections 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48d01084c0 0x7f48d01a5bb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f48d0102480 0x7f48d019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f48b80b2880 0x7f48b80b4d40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 --2- 172.21.15.47:0/3414447974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f48d0102e40 0x7f48d019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 >> 172.21.15.47:0/3414447974 conn(0x7f48d0079400 msgr2=0x7f48d0100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 shutdown_connections 2024-05-11T12:21:30.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:30.629+0000 7f48d5281700 1 -- 172.21.15.47:0/3414447974 wait complete. 2024-05-11T12:21:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:30 smithi047 ceph-mon[30375]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2024-05-11T12:21:30.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:30 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3414447974' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-11T12:21:30.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:30 smithi050 ceph-mon[34072]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2024-05-11T12:21:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:30 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3414447974' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-11T12:21:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:30 smithi064 ceph-mon[34537]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 85 B/s wr, 0 op/s 2024-05-11T12:21:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:30 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3414447974' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-11T12:21:31.357 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-11T12:21:31.357 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-11T12:21:31.615 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:31.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:31 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3414447974' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-11T12:21:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:31 smithi050 ceph-mon[34072]: fsmap cephfs:1 {0=h=up:active} 1 up:standby-replay 10 up:standby 2024-05-11T12:21:31.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:31 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3414447974' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-11T12:21:31.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:31 smithi047 ceph-mon[30375]: fsmap cephfs:1 {0=h=up:active} 1 up:standby-replay 10 up:standby 2024-05-11T12:21:32.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:31 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3414447974' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-11T12:21:32.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:31 smithi064 ceph-mon[34537]: fsmap cephfs:1 {0=h=up:active} 1 up:standby-replay 10 up:standby 2024-05-11T12:21:32.924 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:32 smithi047 ceph-mon[30375]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 0 op/s 2024-05-11T12:21:32.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:32 smithi050 ceph-mon[34072]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 0 op/s 2024-05-11T12:21:33.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:32 smithi064 ceph-mon[34537]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 0 op/s 2024-05-11T12:21:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 -- 172.21.15.47:0/3048018987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 msgr2=0x7f65f806c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 --2- 172.21.15.47:0/3048018987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f806c0e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009990 tx=0x7f65e8009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 -- 172.21.15.47:0/3048018987 shutdown_connections 2024-05-11T12:21:33.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 --2- 172.21.15.47:0/3048018987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65f810ffa0 0x7f65f8112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 --2- 172.21.15.47:0/3048018987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f806c0e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 --2- 172.21.15.47:0/3048018987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65f806c620 0x7f65f810fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 -- 172.21.15.47:0/3048018987 >> 172.21.15.47:0/3048018987 conn(0x7f65f80794f0 msgr2=0x7f65f8079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 -- 172.21.15.47:0/3048018987 shutdown_connections 2024-05-11T12:21:33.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.182+0000 7f65fde1a700 1 -- 172.21.15.47:0/3048018987 wait complete. 2024-05-11T12:21:33.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 Processor -- start 2024-05-11T12:21:33.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 -- start start 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65f806c620 0x7f65f819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65f810ffa0 0x7f65f81a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8115970 con 0x7f65f806bd00 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81157f0 con 0x7f65f810ffa0 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65fde1a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8115af0 con 0x7f65f806c620 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44966/0 (socket says 172.21.15.47:44966) 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f77fe700 1 -- 172.21.15.47:0/991918044 learned_addr learned my addr 172.21.15.47:0/991918044 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f6ffd700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65f806c620 0x7f65f819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f77fe700 1 -- 172.21.15.47:0/991918044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65f806c620 msgr2=0x7f65f819c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f77fe700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65f806c620 0x7f65f819c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f77fe700 1 -- 172.21.15.47:0/991918044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65f810ffa0 msgr2=0x7f65f81a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:33.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.184+0000 7f65f77fe700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65f810ffa0 0x7f65f81a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.185+0000 7f65f77fe700 1 -- 172.21.15.47:0/991918044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f81a0e70 con 0x7f65f806bd00 2024-05-11T12:21:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.185+0000 7f65f77fe700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f819be60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000c00 tx=0x7f65e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.185+0000 7f65f4ff9700 1 -- 172.21.15.47:0/991918044 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8027070 con 0x7f65f806bd00 2024-05-11T12:21:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.185+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f81a1100 con 0x7f65f806bd00 2024-05-11T12:21:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.185+0000 7f65f4ff9700 1 -- 172.21.15.47:0/991918044 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e80041d0 con 0x7f65f806bd00 2024-05-11T12:21:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.185+0000 7f65f4ff9700 1 -- 172.21.15.47:0/991918044 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e800f460 con 0x7f65f806bd00 2024-05-11T12:21:33.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.185+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f81a89e0 con 0x7f65f806bd00 2024-05-11T12:21:33.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.186+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f806a240 con 0x7f65f806bd00 2024-05-11T12:21:33.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.187+0000 7f65f4ff9700 1 -- 172.21.15.47:0/991918044 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f65e802a070 con 0x7f65f806bd00 2024-05-11T12:21:33.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.188+0000 7f65f4ff9700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f65e00b2880 0x7f65e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.188+0000 7f65f4ff9700 1 -- 172.21.15.47:0/991918044 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f65e80e3500 con 0x7f65f806bd00 2024-05-11T12:21:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.188+0000 7f65f6ffd700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f65e00b2880 0x7f65e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:33.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.189+0000 7f65f6ffd700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f65e00b2880 0x7f65e00b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65f8103a50 tx=0x7f65ec005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:33.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.191+0000 7f65f4ff9700 1 -- 172.21.15.47:0/991918044 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65e8031070 con 0x7f65f806bd00 2024-05-11T12:21:33.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.435+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f65f8063990 con 0x7f65e00b2880 2024-05-11T12:21:33.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.458+0000 7f65f4ff9700 1 -- 172.21.15.47:0/991918044 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65f8063990 con 0x7f65e00b2880 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f65e00b2880 msgr2=0x7f65e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f65e00b2880 0x7f65e00b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65f8103a50 tx=0x7f65ec005eb0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 msgr2=0x7f65f819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f819be60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f65e8000c00 tx=0x7f65e80047b0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 shutdown_connections 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f65e00b2880 0x7f65e00b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f65f810ffa0 0x7f65f81a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65f806bd00 0x7f65f819be60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 --2- 172.21.15.47:0/991918044 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f65f806c620 0x7f65f819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 >> 172.21.15.47:0/991918044 conn(0x7f65f80794f0 msgr2=0x7f65f8111630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 shutdown_connections 2024-05-11T12:21:33.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:33.461+0000 7f65fde1a700 1 -- 172.21.15.47:0/991918044 wait complete. 2024-05-11T12:21:33.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:21:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:21:33.441+0000 7fb74fb11700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T12:21:33.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:21:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:21:33.441+0000 7fb74fb11700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T12:21:33.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:21:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:21:33.441+0000 7fb74fb11700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T12:21:33.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:21:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:21:33.441+0000 7fb74fb11700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T12:21:33.775 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:21:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:21:33.441+0000 7fb74fb11700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-11T12:21:34.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-11T12:21:34.318 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:34.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:34 smithi047 ceph-mon[30375]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:34.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:34 smithi047 ceph-mon[30375]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:21:34.694 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:34 smithi047 ceph-mon[30375]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-11T12:21:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:34 smithi050 ceph-mon[34072]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:34 smithi050 ceph-mon[34072]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:21:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:34 smithi050 ceph-mon[34072]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-11T12:21:35.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:34 smithi064 ceph-mon[34537]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:35.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:34 smithi064 ceph-mon[34537]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:21:35.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:34 smithi064 ceph-mon[34537]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-11T12:21:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 -- 172.21.15.47:0/4072678433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 msgr2=0x7fa27406c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 --2- 172.21.15.47:0/4072678433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27406c300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b0d0 tx=0x7fa27000b3e0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 -- 172.21.15.47:0/4072678433 shutdown_connections 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 --2- 172.21.15.47:0/4072678433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa274108870 0x7fa274110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 --2- 172.21.15.47:0/4072678433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27406c300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 --2- 172.21.15.47:0/4072678433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27406c8d0 0x7fa274108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 -- 172.21.15.47:0/4072678433 >> 172.21.15.47:0/4072678433 conn(0x7fa2740784a0 msgr2=0x7fa2740788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 -- 172.21.15.47:0/4072678433 shutdown_connections 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.075+0000 7fa27b90d700 1 -- 172.21.15.47:0/4072678433 wait complete. 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 Processor -- start 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 -- start start 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27406c8d0 0x7fa27419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:36.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa274108870 0x7fa274199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa2741137a0 con 0x7fa27406bf20 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa274113620 con 0x7fa27406c8d0 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa27b90d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa274113920 con 0x7fa274108870 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa2796a9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa2796a9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44990/0 (socket says 172.21.15.47:44990) 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.076+0000 7fa2796a9700 1 -- 172.21.15.47:0/2163965210 learned_addr learned my addr 172.21.15.47:0/2163965210 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa279eaa700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa274108870 0x7fa274199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa278ea8700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27406c8d0 0x7fa27419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa2796a9700 1 -- 172.21.15.47:0/2163965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa274108870 msgr2=0x7fa274199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa2796a9700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa274108870 0x7fa274199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa2796a9700 1 -- 172.21.15.47:0/2163965210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27406c8d0 msgr2=0x7fa27419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa2796a9700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27406c8d0 0x7fa27419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa2796a9700 1 -- 172.21.15.47:0/2163965210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27419a180 con 0x7fa27406bf20 2024-05-11T12:21:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa2796a9700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27419f810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d010 tx=0x7fa270007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa26a7fc700 1 -- 172.21.15.47:0/2163965210 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27001f070 con 0x7fa27406bf20 2024-05-11T12:21:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa26a7fc700 1 -- 172.21.15.47:0/2163965210 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa270003c00 con 0x7fa27406bf20 2024-05-11T12:21:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27419a410 con 0x7fa27406bf20 2024-05-11T12:21:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa26a7fc700 1 -- 172.21.15.47:0/2163965210 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27001b6a0 con 0x7fa27406bf20 2024-05-11T12:21:36.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.077+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2741a66e0 con 0x7fa27406bf20 2024-05-11T12:21:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.078+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27406a240 con 0x7fa27406bf20 2024-05-11T12:21:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.079+0000 7fa26a7fc700 1 -- 172.21.15.47:0/2163965210 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa270022070 con 0x7fa27406bf20 2024-05-11T12:21:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.080+0000 7fa26a7fc700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa2600b2a10 0x7fa2600b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.080+0000 7fa278ea8700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa2600b2a10 0x7fa2600b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.080+0000 7fa26a7fc700 1 -- 172.21.15.47:0/2163965210 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fa27000f020 con 0x7fa27406bf20 2024-05-11T12:21:36.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.081+0000 7fa278ea8700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa2600b2a10 0x7fa2600b4ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa264005950 tx=0x7fa2640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:36.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.084+0000 7fa26a7fc700 1 -- 172.21.15.47:0/2163965210 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa270029020 con 0x7fa27406bf20 2024-05-11T12:21:36.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.332+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa274063990 con 0x7fa2600b2a10 2024-05-11T12:21:36.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.365+0000 7fa26a7fc700 1 -- 172.21.15.47:0/2163965210 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa274063990 con 0x7fa2600b2a10 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa2600b2a10 msgr2=0x7fa2600b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa2600b2a10 0x7fa2600b4ed0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa264005950 tx=0x7fa2640058e0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 msgr2=0x7fa27419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27419f810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa27000d010 tx=0x7fa270007ad0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 shutdown_connections 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa274108870 0x7fa274199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa27406bf20 0x7fa27419f810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa2600b2a10 0x7fa2600b4ed0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 --2- 172.21.15.47:0/2163965210 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa27406c8d0 0x7fa27419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 >> 172.21.15.47:0/2163965210 conn(0x7fa2740784a0 msgr2=0x7fa274100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 shutdown_connections 2024-05-11T12:21:36.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:36.367+0000 7fa27b90d700 1 -- 172.21.15.47:0/2163965210 wait complete. 2024-05-11T12:21:36.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:36 smithi050 ceph-mon[34072]: pgmap v306: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-11T12:21:37.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:36 smithi047 ceph-mon[30375]: pgmap v306: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-11T12:21:37.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-11T12:21:37.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:36 smithi064 ceph-mon[34537]: pgmap v306: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-11T12:21:37.344 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:37.722 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:37 smithi047 ceph-mon[30375]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:37.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:37 smithi050 ceph-mon[34072]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:38.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:37 smithi064 ceph-mon[34537]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:38.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:38 smithi047 ceph-mon[30375]: pgmap v307: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-11T12:21:38.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:38 smithi050 ceph-mon[34072]: pgmap v307: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-11T12:21:39.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 -- 172.21.15.47:0/3043177204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c06c6e0 msgr2=0x7f121c110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:39.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 --2- 172.21.15.47:0/3043177204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c06c6e0 0x7f121c110410 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f121400b0d0 tx=0x7f121400b3e0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 -- 172.21.15.47:0/3043177204 shutdown_connections 2024-05-11T12:21:39.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 --2- 172.21.15.47:0/3043177204 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c06c6e0 0x7f121c110410 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 --2- 172.21.15.47:0/3043177204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f121c109640 0x7f121c109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 --2- 172.21.15.47:0/3043177204 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f121c06bd20 0x7f121c06c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 -- 172.21.15.47:0/3043177204 >> 172.21.15.47:0/3043177204 conn(0x7f121c0794f0 msgr2=0x7f121c079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:39.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 -- 172.21.15.47:0/3043177204 shutdown_connections 2024-05-11T12:21:39.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.047+0000 7f1222611700 1 -- 172.21.15.47:0/3043177204 wait complete. 2024-05-11T12:21:39.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 Processor -- start 2024-05-11T12:21:39.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 -- start start 2024-05-11T12:21:39.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f121c06bd20 0x7f121c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:39.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f121c06c6e0 0x7f121c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:39.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c109640 0x7f121c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f121c113760 con 0x7f121c109640 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1135e0 con 0x7f121c06c6e0 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.048+0000 7f1222611700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f121c1138e0 con 0x7f121c06bd20 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c109640 0x7f121c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c109640 0x7f121c199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46382/0 (socket says 172.21.15.47:46382) 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 -- 172.21.15.47:0/2558444215 learned_addr learned my addr 172.21.15.47:0/2558444215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1213fff700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f121c06c6e0 0x7f121c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f121bfff700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f121c06bd20 0x7f121c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 -- 172.21.15.47:0/2558444215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f121c06bd20 msgr2=0x7f121c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f121c06bd20 0x7f121c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 -- 172.21.15.47:0/2558444215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f121c06c6e0 msgr2=0x7f121c19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f121c06c6e0 0x7f121c19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 -- 172.21.15.47:0/2558444215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121c19a160 con 0x7f121c109640 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1220bae700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c109640 0x7f121c199a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12140095c0 tx=0x7f1214007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1219ffb700 1 -- 172.21.15.47:0/2558444215 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121401f070 con 0x7f121c109640 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1219ffb700 1 -- 172.21.15.47:0/2558444215 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1214007e20 con 0x7f121c109640 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121c19a3f0 con 0x7f121c109640 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1219ffb700 1 -- 172.21.15.47:0/2558444215 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121401b4a0 con 0x7f121c109640 2024-05-11T12:21:39.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.049+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f121c1a6710 con 0x7f121c109640 2024-05-11T12:21:39.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.051+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121c104cb0 con 0x7f121c109640 2024-05-11T12:21:39.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.052+0000 7f1219ffb700 1 -- 172.21.15.47:0/2558444215 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1214022070 con 0x7f121c109640 2024-05-11T12:21:39.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.052+0000 7f1219ffb700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f12080b2910 0x7f12080b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:39.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.053+0000 7f121bfff700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f12080b2910 0x7f12080b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:39.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.053+0000 7f1219ffb700 1 -- 172.21.15.47:0/2558444215 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7f121400f020 con 0x7f121c109640 2024-05-11T12:21:39.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.053+0000 7f121bfff700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f12080b2910 0x7f12080b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f121c19ab40 tx=0x7f120c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:39.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.057+0000 7f1219ffb700 1 -- 172.21.15.47:0/2558444215 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1214029020 con 0x7f121c109640 2024-05-11T12:21:39.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:38 smithi064 ceph-mon[34537]: pgmap v307: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 853 B/s wr, 2 op/s 2024-05-11T12:21:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.305+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f121c063990 con 0x7f12080b2910 2024-05-11T12:21:39.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.311+0000 7f1219ffb700 1 -- 172.21.15.47:0/2558444215 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f121c063990 con 0x7f12080b2910 2024-05-11T12:21:39.311 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_0/a4d3cd26-0138-418e-ad48-d0f5743e805b 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f12080b2910 msgr2=0x7f12080b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f12080b2910 0x7f12080b4dd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f121c19ab40 tx=0x7f120c00b410 comp rx=0 tx=0).stop 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c109640 msgr2=0x7f121c199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c109640 0x7f121c199a00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f12140095c0 tx=0x7f1214007a20 comp rx=0 tx=0).stop 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 shutdown_connections 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f121c109640 0x7f121c199a00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f121c06bd20 0x7f121c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f12080b2910 0x7f12080b4dd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 --2- 172.21.15.47:0/2558444215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f121c06c6e0 0x7f121c19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 >> 172.21.15.47:0/2558444215 conn(0x7f121c0794f0 msgr2=0x7f121c101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 shutdown_connections 2024-05-11T12:21:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:39.314+0000 7f1222611700 1 -- 172.21.15.47:0/2558444215 wait complete. 2024-05-11T12:21:39.971 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-11T12:21:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:40.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:40 smithi050 ceph-mon[34072]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:40 smithi050 ceph-mon[34072]: pgmap v308: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-11T12:21:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:40 smithi047 ceph-mon[30375]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:40 smithi047 ceph-mon[30375]: pgmap v308: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-11T12:21:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:40 smithi064 ceph-mon[34537]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:40 smithi064 ceph-mon[34537]: pgmap v308: 97 pgs: 97 active+clean; 585 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-11T12:21:41.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 -- 172.21.15.47:0/255887160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 msgr2=0x7f009c110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 --2- 172.21.15.47:0/255887160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c110480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f00940099c0 tx=0x7f0094009cd0 comp rx=0 tx=0).stop 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 -- 172.21.15.47:0/255887160 shutdown_connections 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 --2- 172.21.15.47:0/255887160 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c110480 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 --2- 172.21.15.47:0/255887160 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f009c103800 0x7f009c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 --2- 172.21.15.47:0/255887160 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f009c1041b0 0x7f009c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 -- 172.21.15.47:0/255887160 >> 172.21.15.47:0/255887160 conn(0x7f009c07a6b0 msgr2=0x7f009c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 -- 172.21.15.47:0/255887160 shutdown_connections 2024-05-11T12:21:41.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.892+0000 7f00a46e5700 1 -- 172.21.15.47:0/255887160 wait complete. 2024-05-11T12:21:41.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 4]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-11T12:21:41.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-05-11T12:21:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 Processor -- start 2024-05-11T12:21:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 -- start start 2024-05-11T12:21:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f009c103800 0x7f009c1a4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f009c1041b0 0x7f009c1a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c10f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:41.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f009c1136d0 con 0x7f009c108900 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f009c113550 con 0x7f009c103800 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.893+0000 7f00a46e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f009c113850 con 0x7f009c1041b0 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c10f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c10f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46400/0 (socket says 172.21.15.47:46400) 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 -- 172.21.15.47:0/851416377 learned_addr learned my addr 172.21.15.47:0/851416377 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a1c80700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f009c1041b0 0x7f009c1a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2481700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f009c103800 0x7f009c1a4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 -- 172.21.15.47:0/851416377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f009c1041b0 msgr2=0x7f009c1a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f009c1041b0 0x7f009c1a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 -- 172.21.15.47:0/851416377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f009c103800 msgr2=0x7f009c1a4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f009c103800 0x7f009c1a4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 -- 172.21.15.47:0/851416377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c10fd70 con 0x7f009c108900 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00a2c82700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c10f5e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0094005910 tx=0x7f0094009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:41.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.894+0000 7f00937fe700 1 -- 172.21.15.47:0/851416377 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094027070 con 0x7f009c108900 2024-05-11T12:21:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.895+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009c110000 con 0x7f009c108900 2024-05-11T12:21:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.895+0000 7f00937fe700 1 -- 172.21.15.47:0/851416377 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0094003fa0 con 0x7f009c108900 2024-05-11T12:21:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.895+0000 7f00937fe700 1 -- 172.21.15.47:0/851416377 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0094022d10 con 0x7f009c108900 2024-05-11T12:21:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.895+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009c1acc50 con 0x7f009c108900 2024-05-11T12:21:41.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.896+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c104c90 con 0x7f009c108900 2024-05-11T12:21:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.897+0000 7f00937fe700 1 -- 172.21.15.47:0/851416377 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f009402a070 con 0x7f009c108900 2024-05-11T12:21:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.898+0000 7f00937fe700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f00880b2910 0x7f00880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.898+0000 7f00a2481700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f00880b2910 0x7f00880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.898+0000 7f00937fe700 1 -- 172.21.15.47:0/851416377 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8269+0+0 (secure 0 0 0) 0x7f00940e3270 con 0x7f009c108900 2024-05-11T12:21:41.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.898+0000 7f00a2481700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f00880b2910 0x7f00880b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0098009d10 tx=0x7f0098009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:41.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:41.901+0000 7f00937fe700 1 -- 172.21.15.47:0/851416377 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0094013070 con 0x7f009c108900 2024-05-11T12:21:41.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 4]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-11T12:21:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-05-11T12:21:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-05-11T12:21:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 4]}]: dispatch 2024-05-11T12:21:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-11T12:21:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-05-11T12:21:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-05-11T12:21:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-05-11T12:21:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]: dispatch 2024-05-11T12:21:42.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]: dispatch 2024-05-11T12:21:42.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]: dispatch 2024-05-11T12:21:42.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]: dispatch 2024-05-11T12:21:42.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.146+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f009c101bf0 con 0x7f00880b2910 2024-05-11T12:21:42.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.172+0000 7f00937fe700 1 -- 172.21.15.47:0/851416377 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f009c101bf0 con 0x7f00880b2910 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f00880b2910 msgr2=0x7f00880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f00880b2910 0x7f00880b4dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0098009d10 tx=0x7f0098009040 comp rx=0 tx=0).stop 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 msgr2=0x7f009c10f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c10f5e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0094005910 tx=0x7f0094009580 comp rx=0 tx=0).stop 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 shutdown_connections 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f009c108900 0x7f009c10f5e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f00880b2910 0x7f00880b4dd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f009c103800 0x7f009c1a4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 --2- 172.21.15.47:0/851416377 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f009c1041b0 0x7f009c1a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:42.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 >> 172.21.15.47:0/851416377 conn(0x7f009c07a6b0 msgr2=0x7f009c1014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:42.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 shutdown_connections 2024-05-11T12:21:42.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:42.174+0000 7f00a46e5700 1 -- 172.21.15.47:0/851416377 wait complete. 2024-05-11T12:21:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: pgmap v309: 97 pgs: 97 active+clean; 605 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-05-11T12:21:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-11T12:21:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 4]}]': finished 2024-05-11T12:21:42.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-05-11T12:21:42.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:42 smithi047 ceph-mon[30375]: osdmap e80: 12 total, 12 up, 12 in 2024-05-11T12:21:42.806 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: pgmap v309: 97 pgs: 97 active+clean; 605 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 4]}]': finished 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]': finished 2024-05-11T12:21:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]': finished 2024-05-11T12:21:42.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-11T12:21:42.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-05-11T12:21:42.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:42 smithi050 ceph-mon[34072]: osdmap e80: 12 total, 12 up, 12 in 2024-05-11T12:21:43.063 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: pgmap v309: 97 pgs: 97 active+clean; 605 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.8 KiB/s wr, 4 op/s 2024-05-11T12:21:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-05-11T12:21:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 4]}]': finished 2024-05-11T12:21:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-11T12:21:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-05-11T12:21:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-05-11T12:21:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-05-11T12:21:43.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.17", "id": [5, 1]}]': finished 2024-05-11T12:21:43.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 6]}]': finished 2024-05-11T12:21:43.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 8]}]': finished 2024-05-11T12:21:43.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 6]}]': finished 2024-05-11T12:21:43.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:42 smithi064 ceph-mon[34537]: osdmap e80: 12 total, 12 up, 12 in 2024-05-11T12:21:43.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:43 smithi047 ceph-mon[30375]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:43.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:43 smithi047 ceph-mon[30375]: osdmap e81: 12 total, 12 up, 12 in 2024-05-11T12:21:43.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:43 smithi050 ceph-mon[34072]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:43 smithi050 ceph-mon[34072]: osdmap e81: 12 total, 12 up, 12 in 2024-05-11T12:21:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:43 smithi064 ceph-mon[34537]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:43 smithi064 ceph-mon[34537]: osdmap e81: 12 total, 12 up, 12 in 2024-05-11T12:21:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.690+0000 7f1589822700 1 -- 172.21.15.47:0/2372942114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15841084c0 msgr2=0x7f15841088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.690+0000 7f1589822700 1 --2- 172.21.15.47:0/2372942114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15841084c0 0x7f15841088a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1574009990 tx=0x7f1574009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.690+0000 7f1589822700 1 -- 172.21.15.47:0/2372942114 shutdown_connections 2024-05-11T12:21:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 --2- 172.21.15.47:0/2372942114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1584102e40 0x7f158410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 --2- 172.21.15.47:0/2372942114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15841084c0 0x7f15841088a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 --2- 172.21.15.47:0/2372942114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1584102480 0x7f1584102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 -- 172.21.15.47:0/2372942114 >> 172.21.15.47:0/2372942114 conn(0x7f1584079400 msgr2=0x7f1584079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 -- 172.21.15.47:0/2372942114 shutdown_connections 2024-05-11T12:21:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 -- 172.21.15.47:0/2372942114 wait complete. 2024-05-11T12:21:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 Processor -- start 2024-05-11T12:21:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.691+0000 7f1589822700 1 -- start start 2024-05-11T12:21:44.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1589822700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1584102480 0x7f1584104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:44.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1589822700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1584102e40 0x7f15841045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:44.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1589822700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15841084c0 0x7f1584104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1589822700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1584112670 con 0x7f1584102480 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1589822700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f15841124f0 con 0x7f15841084c0 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1589822700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f15841127f0 con 0x7f1584102e40 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1584102480 0x7f1584104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1584102480 0x7f1584104090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46418/0 (socket says 172.21.15.47:46418) 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 -- 172.21.15.47:0/1238479556 learned_addr learned my addr 172.21.15.47:0/1238479556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f157a7fc700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1584102e40 0x7f15841045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f15837fe700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15841084c0 0x7f1584104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 -- 172.21.15.47:0/1238479556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1584102e40 msgr2=0x7f15841045d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1584102e40 0x7f15841045d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 -- 172.21.15.47:0/1238479556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15841084c0 msgr2=0x7f1584104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15841084c0 0x7f1584104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 -- 172.21.15.47:0/1238479556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15841a8240 con 0x7f1584102480 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.692+0000 7f1582ffd700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1584102480 0x7f1584104090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1574000c00 tx=0x7f15740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.693+0000 7f1588820700 1 -- 172.21.15.47:0/1238479556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574027070 con 0x7f1584102480 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.693+0000 7f1588820700 1 -- 172.21.15.47:0/1238479556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1574003f20 con 0x7f1584102480 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.693+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15841a84d0 con 0x7f1584102480 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.693+0000 7f1588820700 1 -- 172.21.15.47:0/1238479556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1574022ca0 con 0x7f1584102480 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.693+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15841a8980 con 0x7f1584102480 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.694+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158406a240 con 0x7f1584102480 2024-05-11T12:21:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.695+0000 7f1588820700 1 -- 172.21.15.47:0/1238479556 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f157402a070 con 0x7f1584102480 2024-05-11T12:21:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.697+0000 7f1588820700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15700b2990 0x7f15700b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.697+0000 7f157a7fc700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15700b2990 0x7f15700b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.697+0000 7f1588820700 1 -- 172.21.15.47:0/1238479556 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f15740e38a0 con 0x7f1584102480 2024-05-11T12:21:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.697+0000 7f157a7fc700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15700b2990 0x7f15700b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1584079b90 tx=0x7f156c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:44.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.700+0000 7f1588820700 1 -- 172.21.15.47:0/1238479556 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1574031070 con 0x7f1584102480 2024-05-11T12:21:44.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:44 smithi050 ceph-mon[34072]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 605 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-05-11T12:21:44.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.951+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1584063990 con 0x7f15700b2990 2024-05-11T12:21:44.951 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:44 smithi047 ceph-mon[30375]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 605 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-05-11T12:21:44.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.956+0000 7f1588820700 1 -- 172.21.15.47:0/1238479556 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1584063990 con 0x7f15700b2990 2024-05-11T12:21:44.956 INFO:teuthology.orchestra.run.smithi047.stdout:/volumes/qa/sv_1/e44abbf6-31a0-46a6-8414-5003f221d662 2024-05-11T12:21:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.958+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15700b2990 msgr2=0x7f15700b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.958+0000 7f1589822700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15700b2990 0x7f15700b4e50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1584079b90 tx=0x7f156c009190 comp rx=0 tx=0).stop 2024-05-11T12:21:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.958+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1584102480 msgr2=0x7f1584104090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:44.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.958+0000 7f1589822700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1584102480 0x7f1584104090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1574000c00 tx=0x7f15740047b0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 shutdown_connections 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f15700b2990 0x7f15700b4e50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f15841084c0 0x7f1584104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1584102480 0x7f1584104090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 --2- 172.21.15.47:0/1238479556 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1584102e40 0x7f15841045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 >> 172.21.15.47:0/1238479556 conn(0x7f1584079400 msgr2=0x7f158406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 shutdown_connections 2024-05-11T12:21:44.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:44.959+0000 7f1589822700 1 -- 172.21.15.47:0/1238479556 wait complete. 2024-05-11T12:21:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:44 smithi064 ceph-mon[34537]: pgmap v312: 97 pgs: 3 peering, 94 active+clean; 605 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-05-11T12:21:45.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:21:45.882 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:45.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:45 smithi050 ceph-mon[34072]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:46.005 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:45 smithi047 ceph-mon[30375]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:45 smithi064 ceph-mon[34537]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:21:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:46 smithi050 ceph-mon[34072]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 629 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-11T12:21:47.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:46 smithi047 ceph-mon[30375]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 629 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-11T12:21:47.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:46 smithi064 ceph-mon[34537]: pgmap v313: 97 pgs: 3 peering, 94 active+clean; 629 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.5 KiB/s wr, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-11T12:21:47.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 -- 172.21.15.47:0/3660048376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758108630 msgr2=0x7f7758108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:47.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 --2- 172.21.15.47:0/3660048376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758108630 0x7f7758108a10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f77480098f0 tx=0x7f7748009c00 comp rx=0 tx=0).stop 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 -- 172.21.15.47:0/3660048376 shutdown_connections 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 --2- 172.21.15.47:0/3660048376 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77581028d0 0x7f7758110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 --2- 172.21.15.47:0/3660048376 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758108630 0x7f7758108a10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 --2- 172.21.15.47:0/3660048376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7758101f10 0x7f7758102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 -- 172.21.15.47:0/3660048376 >> 172.21.15.47:0/3660048376 conn(0x7f7758077770 msgr2=0x7f7758077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 -- 172.21.15.47:0/3660048376 shutdown_connections 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.597+0000 7f775e6a4700 1 -- 172.21.15.47:0/3660048376 wait complete. 2024-05-11T12:21:47.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.598+0000 7f775e6a4700 1 Processor -- start 2024-05-11T12:21:47.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.598+0000 7f775e6a4700 1 -- start start 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f775e6a4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758101f10 0x7f775819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f775e6a4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77581028d0 0x7f775819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f775e6a4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7758108630 0x7f7758199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f775e6a4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7758113770 con 0x7f7758101f10 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f775e6a4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f77581135f0 con 0x7f7758108630 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f775e6a4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f77581138f0 con 0x7f77581028d0 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758101f10 0x7f775819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758101f10 0x7f775819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45560/0 (socket says 172.21.15.47:45560) 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 -- 172.21.15.47:0/1026743484 learned_addr learned my addr 172.21.15.47:0/1026743484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f775cc41700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7758108630 0x7f7758199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:47.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f77577fe700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77581028d0 0x7f775819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 -- 172.21.15.47:0/1026743484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77581028d0 msgr2=0x7f775819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77581028d0 0x7f775819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 -- 172.21.15.47:0/1026743484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7758108630 msgr2=0x7f7758199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7758108630 0x7f7758199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 -- 172.21.15.47:0/1026743484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775819a230 con 0x7f7758101f10 2024-05-11T12:21:47.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.599+0000 7f7757fff700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758101f10 0x7f775819f910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7748009230 tx=0x7f774800f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.600+0000 7f77557fa700 1 -- 172.21.15.47:0/1026743484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77480050c0 con 0x7f7758101f10 2024-05-11T12:21:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.600+0000 7f77557fa700 1 -- 172.21.15.47:0/1026743484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774800b560 con 0x7f7758101f10 2024-05-11T12:21:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.600+0000 7f77557fa700 1 -- 172.21.15.47:0/1026743484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7748003a40 con 0x7f7758101f10 2024-05-11T12:21:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.600+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775819a4c0 con 0x7f7758101f10 2024-05-11T12:21:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.600+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77581a67e0 con 0x7f7758101f10 2024-05-11T12:21:47.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.601+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775806b700 con 0x7f7758101f10 2024-05-11T12:21:47.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.602+0000 7f77557fa700 1 -- 172.21.15.47:0/1026743484 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f774800ba10 con 0x7f7758101f10 2024-05-11T12:21:47.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.603+0000 7f77557fa700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77400b2990 0x7f77400b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:47.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.603+0000 7f77577fe700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77400b2990 0x7f77400b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:47.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.603+0000 7f77557fa700 1 -- 172.21.15.47:0/1026743484 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f77480e3600 con 0x7f7758101f10 2024-05-11T12:21:47.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.604+0000 7f77577fe700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77400b2990 0x7f77400b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7758077e70 tx=0x7f774c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:47.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.607+0000 7f77557fa700 1 -- 172.21.15.47:0/1026743484 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7748013030 con 0x7f7758101f10 2024-05-11T12:21:47.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.875+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f775819afc0 con 0x7f7758101f10 2024-05-11T12:21:47.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.876+0000 7f77557fa700 1 -- 172.21.15.47:0/1026743484 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f77480ab230 con 0x7f7758101f10 2024-05-11T12:21:47.877 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:47.877 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:21:47.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77400b2990 msgr2=0x7f77400b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77400b2990 0x7f77400b4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7758077e70 tx=0x7f774c009260 comp rx=0 tx=0).stop 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758101f10 msgr2=0x7f775819f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758101f10 0x7f775819f910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7748009230 tx=0x7f774800f990 comp rx=0 tx=0).stop 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 shutdown_connections 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77400b2990 0x7f77400b4e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7758108630 0x7f7758199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7758101f10 0x7f775819f910 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 --2- 172.21.15.47:0/1026743484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f77581028d0 0x7f775819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 >> 172.21.15.47:0/1026743484 conn(0x7f7758077770 msgr2=0x7f7758100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:47.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 shutdown_connections 2024-05-11T12:21:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:47.879+0000 7f775e6a4700 1 -- 172.21.15.47:0/1026743484 wait complete. 2024-05-11T12:21:47.880 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:21:48.551 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:21:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:48 smithi047 ceph-mon[30375]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 629 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 5 op/s; 4 B/s, 1 keys/s, 1 objects/s recovering 2024-05-11T12:21:48.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:48 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1026743484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:21:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:48.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:48 smithi050 ceph-mon[34072]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 629 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 5 op/s; 4 B/s, 1 keys/s, 1 objects/s recovering 2024-05-11T12:21:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:48 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1026743484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:21:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:48 smithi064 ceph-mon[34537]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 629 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.4 KiB/s wr, 5 op/s; 4 B/s, 1 keys/s, 1 objects/s recovering 2024-05-11T12:21:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:48 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1026743484' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:21:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 -- 172.21.15.47:0/1824502766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 msgr2=0x7f0d60108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 --2- 172.21.15.47:0/1824502766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60108900 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009990 tx=0x7f0d50009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 -- 172.21.15.47:0/1824502766 shutdown_connections 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 --2- 172.21.15.47:0/1824502766 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d60102e00 0x7f0d6010f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 --2- 172.21.15.47:0/1824502766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60108900 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 --2- 172.21.15.47:0/1824502766 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d60102440 0x7f0d601028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 -- 172.21.15.47:0/1824502766 >> 172.21.15.47:0/1824502766 conn(0x7f0d600793c0 msgr2=0x7f0d600797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 -- 172.21.15.47:0/1824502766 shutdown_connections 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.395+0000 7f0d6519c700 1 -- 172.21.15.47:0/1824502766 wait complete. 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.396+0000 7f0d6519c700 1 Processor -- start 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.396+0000 7f0d6519c700 1 -- start start 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d6519c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d60102440 0x7f0d6019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d6519c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d60102e00 0x7f0d6019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d6519c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d6519c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601126a0 con 0x7f0d60108520 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d6519c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112520 con 0x7f0d60102440 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d6519c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112820 con 0x7f0d60102e00 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5ed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d60102440 0x7f0d6019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45574/0 (socket says 172.21.15.47:45574) 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 -- 172.21.15.47:0/176830995 learned_addr learned my addr 172.21.15.47:0/176830995 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5e59c700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d60102e00 0x7f0d6019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 -- 172.21.15.47:0/176830995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d60102e00 msgr2=0x7f0d6019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d60102e00 0x7f0d6019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 -- 172.21.15.47:0/176830995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d60102440 msgr2=0x7f0d6019f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d60102440 0x7f0d6019f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 -- 172.21.15.47:0/176830995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6019a230 con 0x7f0d60108520 2024-05-11T12:21:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d5f59e700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60199ad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b970 tx=0x7f0d5800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.397+0000 7f0d47fff700 1 -- 172.21.15.47:0/176830995 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800d860 con 0x7f0d60108520 2024-05-11T12:21:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.398+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6019a520 con 0x7f0d60108520 2024-05-11T12:21:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.398+0000 7f0d47fff700 1 -- 172.21.15.47:0/176830995 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d5800e430 con 0x7f0d60108520 2024-05-11T12:21:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.398+0000 7f0d47fff700 1 -- 172.21.15.47:0/176830995 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58017660 con 0x7f0d60108520 2024-05-11T12:21:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.398+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d601a67e0 con 0x7f0d60108520 2024-05-11T12:21:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.399+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d6006a240 con 0x7f0d60108520 2024-05-11T12:21:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.400+0000 7f0d47fff700 1 -- 172.21.15.47:0/176830995 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0d5800da00 con 0x7f0d60108520 2024-05-11T12:21:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.401+0000 7f0d47fff700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d480b2990 0x7f0d480b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.401+0000 7f0d5ed9d700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d480b2990 0x7f0d480b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.401+0000 7f0d47fff700 1 -- 172.21.15.47:0/176830995 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f0d580e2350 con 0x7f0d60108520 2024-05-11T12:21:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.401+0000 7f0d5ed9d700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d480b2990 0x7f0d480b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009750 tx=0x7f0d50005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:50.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.404+0000 7f0d47fff700 1 -- 172.21.15.47:0/176830995 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0d58013030 con 0x7f0d60108520 2024-05-11T12:21:50.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.624+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d601a6ad0 con 0x7f0d60108520 2024-05-11T12:21:50.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.626+0000 7f0d47fff700 1 -- 172.21.15.47:0/176830995 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22638 (secure 0 0 0) 0x7f0d580a9f70 con 0x7f0d60108520 2024-05-11T12:21:50.627 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:50.627 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":81,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:21:42.658586+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d480b2990 msgr2=0x7f0d480b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d480b2990 0x7f0d480b4e50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009750 tx=0x7f0d50005fd0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 msgr2=0x7f0d60199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60199ad0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d5800b970 tx=0x7f0d5800bc80 comp rx=0 tx=0).stop 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 shutdown_connections 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d60108520 0x7f0d60199ad0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0d480b2990 0x7f0d480b4e50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0d60102440 0x7f0d6019f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 --2- 172.21.15.47:0/176830995 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0d60102e00 0x7f0d6019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 >> 172.21.15.47:0/176830995 conn(0x7f0d600793c0 msgr2=0x7f0d6006bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 shutdown_connections 2024-05-11T12:21:50.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:50.628+0000 7f0d6519c700 1 -- 172.21.15.47:0/176830995 wait complete. 2024-05-11T12:21:50.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:50 smithi050 ceph-mon[34072]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.6 KiB/s wr, 3 op/s; 4 B/s, 1 keys/s, 1 objects/s recovering 2024-05-11T12:21:50.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:50 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/176830995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:51.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:50 smithi047 ceph-mon[30375]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.6 KiB/s wr, 3 op/s; 4 B/s, 1 keys/s, 1 objects/s recovering 2024-05-11T12:21:51.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:50 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/176830995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:51.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:50 smithi064 ceph-mon[34537]: pgmap v315: 97 pgs: 97 active+clean; 629 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.6 KiB/s wr, 3 op/s; 4 B/s, 1 keys/s, 1 objects/s recovering 2024-05-11T12:21:51.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:50 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/176830995' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:21:51.275 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-11T12:21:51.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T12:21:51.529 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:52.915 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:52 smithi047 ceph-mon[30375]: pgmap v316: 97 pgs: 97 active+clean; 634 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.6 KiB/s wr, 4 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-05-11T12:21:52.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:52 smithi050 ceph-mon[34072]: pgmap v316: 97 pgs: 97 active+clean; 634 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.6 KiB/s wr, 4 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-05-11T12:21:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:52 smithi064 ceph-mon[34537]: pgmap v316: 97 pgs: 97 active+clean; 634 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 3.6 KiB/s wr, 4 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-05-11T12:21:53.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 -- 172.21.15.47:0/1186266152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91901095c0 msgr2=0x7f91901099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:53.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 --2- 172.21.15.47:0/1186266152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91901095c0 0x7f91901099a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f918c009990 tx=0x7f918c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 -- 172.21.15.47:0/1186266152 shutdown_connections 2024-05-11T12:21:53.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 --2- 172.21.15.47:0/1186266152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f919006bbf0 0x7f91901103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 --2- 172.21.15.47:0/1186266152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f91901095c0 0x7f91901099a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 --2- 172.21.15.47:0/1186266152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919006b230 0x7f919006b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 -- 172.21.15.47:0/1186266152 >> 172.21.15.47:0/1186266152 conn(0x7f9190079400 msgr2=0x7f9190079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:53.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 -- 172.21.15.47:0/1186266152 shutdown_connections 2024-05-11T12:21:53.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.147+0000 7f91974e9700 1 -- 172.21.15.47:0/1186266152 wait complete. 2024-05-11T12:21:53.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 Processor -- start 2024-05-11T12:21:53.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 -- start start 2024-05-11T12:21:53.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919006b230 0x7f919019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:53.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f919006bbf0 0x7f919019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:53.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91901095c0 0x7f9190199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f91901136d0 con 0x7f919006bbf0 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9190113550 con 0x7f91901095c0 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f91974e9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9190113850 con 0x7f919006b230 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f9194a84700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f919006bbf0 0x7f919019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f9194a84700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f919006bbf0 0x7f919019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45586/0 (socket says 172.21.15.47:45586) 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.148+0000 7f9194a84700 1 -- 172.21.15.47:0/663426517 learned_addr learned my addr 172.21.15.47:0/663426517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9195285700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919006b230 0x7f919019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9195a86700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91901095c0 0x7f9190199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9194a84700 1 -- 172.21.15.47:0/663426517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919006b230 msgr2=0x7f919019f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:53.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9194a84700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919006b230 0x7f919019f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9194a84700 1 -- 172.21.15.47:0/663426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91901095c0 msgr2=0x7f9190199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:53.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9194a84700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91901095c0 0x7f9190199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9194a84700 1 -- 172.21.15.47:0/663426517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919019a0f0 con 0x7f919006bbf0 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f9194a84700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f919006bbf0 0x7f919019fd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f918000c920 tx=0x7f918000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f91867fc700 1 -- 172.21.15.47:0/663426517 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180007b00 con 0x7f919006bbf0 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f91867fc700 1 -- 172.21.15.47:0/663426517 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9180004ce0 con 0x7f919006bbf0 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f91867fc700 1 -- 172.21.15.47:0/663426517 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180005650 con 0x7f919006bbf0 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919019a3e0 con 0x7f919006bbf0 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.149+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901a66a0 con 0x7f919006bbf0 2024-05-11T12:21:53.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.150+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919010baa0 con 0x7f919006bbf0 2024-05-11T12:21:53.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.152+0000 7f91867fc700 1 -- 172.21.15.47:0/663426517 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9180004750 con 0x7f919006bbf0 2024-05-11T12:21:53.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.152+0000 7f91867fc700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f917c0b2990 0x7f917c0b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:53.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.152+0000 7f9195285700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f917c0b2990 0x7f917c0b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:53.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.152+0000 7f91867fc700 1 -- 172.21.15.47:0/663426517 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f91800e2390 con 0x7f919006bbf0 2024-05-11T12:21:53.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.153+0000 7f9195285700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f917c0b2990 0x7f917c0b4e50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f918c006010 tx=0x7f918c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:53.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.156+0000 7f91867fc700 1 -- 172.21.15.47:0/663426517 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9180014030 con 0x7f919006bbf0 2024-05-11T12:21:53.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.366+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9190063990 con 0x7f917c0b2990 2024-05-11T12:21:53.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.373+0000 7f91867fc700 1 -- 172.21.15.47:0/663426517 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258872 (secure 0 0 0) 0x7f9190063990 con 0x7f917c0b2990 2024-05-11T12:21:53.376 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f917c0b2990 msgr2=0x7f917c0b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f917c0b2990 0x7f917c0b4e50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f918c006010 tx=0x7f918c00b540 comp rx=0 tx=0).stop 2024-05-11T12:21:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f919006bbf0 msgr2=0x7f919019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f919006bbf0 0x7f919019fd10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f918000c920 tx=0x7f918000cce0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 shutdown_connections 2024-05-11T12:21:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f917c0b2990 0x7f917c0b4e50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f91901095c0 0x7f9190199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f919006b230 0x7f919019f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 --2- 172.21.15.47:0/663426517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f919006bbf0 0x7f919019fd10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:53.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 >> 172.21.15.47:0/663426517 conn(0x7f9190079400 msgr2=0x7f9190101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:53.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 shutdown_connections 2024-05-11T12:21:53.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:53.377+0000 7f91974e9700 1 -- 172.21.15.47:0/663426517 wait complete. 2024-05-11T12:21:53.382 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-05-11T12:21:53.993 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-05-11T12:21:51.540922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648893,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":136,"num_read_kb":152,"num_write":207,"num_write_kb":2234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951874,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393072,"kb_used_data":10592,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124548752,"statfs":{"total":1151940427776,"available":1151537922048,"internally_reserved":0,"allocated":10846208,"data_stored":7075145,"data_compressed":176607,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29527,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":23,"num_read_kb":40,"num_write":22,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.881379"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662405+0000","last_change":"2024-05-11T12:21:11.536657+0000","last_active":"2024-05-11T12:21:42.662405+0000","last_peered":"2024-05-11T12:21:42.662405+0000","last_clean":"2024-05-11T12:21:42.662405+0000","last_became_active":"2024-05-11T12:21:11.536243+0000","last_became_peered":"2024-05-11T12:21:11.536243+0000","last_unstale":"2024-05-11T12:21:42.662405+0000","last_undegraded":"2024-05-11T12:21:42.662405+0000","last_fullsized":"2024-05-11T12:21:42.662405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:29:20.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665724+0000","last_change":"2024-05-11T12:21:11.527602+0000","last_active":"2024-05-11T12:21:42.665724+0000","last_peered":"2024-05-11T12:21:42.665724+0000","last_clean":"2024-05-11T12:21:42.665724+0000","last_became_active":"2024-05-11T12:21:11.527309+0000","last_became_peered":"2024-05-11T12:21:11.527309+0000","last_unstale":"2024-05-11T12:21:42.665724+0000","last_undegraded":"2024-05-11T12:21:42.665724+0000","last_fullsized":"2024-05-11T12:21:42.665724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:03.764570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665570+0000","last_change":"2024-05-11T12:21:11.529094+0000","last_active":"2024-05-11T12:21:42.665570+0000","last_peered":"2024-05-11T12:21:42.665570+0000","last_clean":"2024-05-11T12:21:42.665570+0000","last_became_active":"2024-05-11T12:21:11.528796+0000","last_became_peered":"2024-05-11T12:21:11.528796+0000","last_unstale":"2024-05-11T12:21:42.665570+0000","last_undegraded":"2024-05-11T12:21:42.665570+0000","last_fullsized":"2024-05-11T12:21:42.665570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:46:41.098066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669710+0000","last_change":"2024-05-11T12:21:11.528903+0000","last_active":"2024-05-11T12:21:42.669710+0000","last_peered":"2024-05-11T12:21:42.669710+0000","last_clean":"2024-05-11T12:21:42.669710+0000","last_became_active":"2024-05-11T12:21:11.528545+0000","last_became_peered":"2024-05-11T12:21:11.528545+0000","last_unstale":"2024-05-11T12:21:42.669710+0000","last_undegraded":"2024-05-11T12:21:42.669710+0000","last_fullsized":"2024-05-11T12:21:42.669710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:56.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665264+0000","last_change":"2024-05-11T12:21:11.521074+0000","last_active":"2024-05-11T12:21:42.665264+0000","last_peered":"2024-05-11T12:21:42.665264+0000","last_clean":"2024-05-11T12:21:42.665264+0000","last_became_active":"2024-05-11T12:21:11.520675+0000","last_became_peered":"2024-05-11T12:21:11.520675+0000","last_unstale":"2024-05-11T12:21:42.665264+0000","last_undegraded":"2024-05-11T12:21:42.665264+0000","last_fullsized":"2024-05-11T12:21:42.665264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:22:50.952687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671243+0000","last_change":"2024-05-11T12:21:42.671243+0000","last_active":"2024-05-11T12:21:42.671243+0000","last_peered":"2024-05-11T12:21:42.671243+0000","last_clean":"2024-05-11T12:21:42.671243+0000","last_became_active":"2024-05-11T12:21:42.670801+0000","last_became_peered":"2024-05-11T12:21:42.670801+0000","last_unstale":"2024-05-11T12:21:42.671243+0000","last_undegraded":"2024-05-11T12:21:42.671243+0000","last_fullsized":"2024-05-11T12:21:42.671243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:02.539855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665943+0000","last_change":"2024-05-11T12:21:11.529961+0000","last_active":"2024-05-11T12:21:42.665943+0000","last_peered":"2024-05-11T12:21:42.665943+0000","last_clean":"2024-05-11T12:21:42.665943+0000","last_became_active":"2024-05-11T12:21:11.529412+0000","last_became_peered":"2024-05-11T12:21:11.529412+0000","last_unstale":"2024-05-11T12:21:42.665943+0000","last_undegraded":"2024-05-11T12:21:42.665943+0000","last_fullsized":"2024-05-11T12:21:42.665943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:00.579881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663513+0000","last_change":"2024-05-11T12:21:11.532874+0000","last_active":"2024-05-11T12:21:42.663513+0000","last_peered":"2024-05-11T12:21:42.663513+0000","last_clean":"2024-05-11T12:21:42.663513+0000","last_became_active":"2024-05-11T12:21:11.532237+0000","last_became_peered":"2024-05-11T12:21:11.532237+0000","last_unstale":"2024-05-11T12:21:42.663513+0000","last_undegraded":"2024-05-11T12:21:42.663513+0000","last_fullsized":"2024-05-11T12:21:42.663513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:13.770719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670901+0000","last_change":"2024-05-11T12:21:11.536465+0000","last_active":"2024-05-11T12:21:42.670901+0000","last_peered":"2024-05-11T12:21:42.670901+0000","last_clean":"2024-05-11T12:21:42.670901+0000","last_became_active":"2024-05-11T12:21:11.536057+0000","last_became_peered":"2024-05-11T12:21:11.536057+0000","last_unstale":"2024-05-11T12:21:42.670901+0000","last_undegraded":"2024-05-11T12:21:42.670901+0000","last_fullsized":"2024-05-11T12:21:42.670901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:22:45.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662835+0000","last_change":"2024-05-11T12:21:11.528187+0000","last_active":"2024-05-11T12:21:42.662835+0000","last_peered":"2024-05-11T12:21:42.662835+0000","last_clean":"2024-05-11T12:21:42.662835+0000","last_became_active":"2024-05-11T12:21:11.527780+0000","last_became_peered":"2024-05-11T12:21:11.527780+0000","last_unstale":"2024-05-11T12:21:42.662835+0000","last_undegraded":"2024-05-11T12:21:42.662835+0000","last_fullsized":"2024-05-11T12:21:42.662835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:23.484372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666391+0000","last_change":"2024-05-11T12:21:11.543388+0000","last_active":"2024-05-11T12:21:42.666391+0000","last_peered":"2024-05-11T12:21:42.666391+0000","last_clean":"2024-05-11T12:21:42.666391+0000","last_became_active":"2024-05-11T12:21:11.542984+0000","last_became_peered":"2024-05-11T12:21:11.542984+0000","last_unstale":"2024-05-11T12:21:42.666391+0000","last_undegraded":"2024-05-11T12:21:42.666391+0000","last_fullsized":"2024-05-11T12:21:42.666391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:52.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.672467+0000","last_change":"2024-05-11T12:21:11.523355+0000","last_active":"2024-05-11T12:21:42.672467+0000","last_peered":"2024-05-11T12:21:42.672467+0000","last_clean":"2024-05-11T12:21:42.672467+0000","last_became_active":"2024-05-11T12:21:11.523007+0000","last_became_peered":"2024-05-11T12:21:11.523007+0000","last_unstale":"2024-05-11T12:21:42.672467+0000","last_undegraded":"2024-05-11T12:21:42.672467+0000","last_fullsized":"2024-05-11T12:21:42.672467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:57.197028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661881+0000","last_change":"2024-05-11T12:21:11.528072+0000","last_active":"2024-05-11T12:21:42.661881+0000","last_peered":"2024-05-11T12:21:42.661881+0000","last_clean":"2024-05-11T12:21:42.661881+0000","last_became_active":"2024-05-11T12:21:11.527478+0000","last_became_peered":"2024-05-11T12:21:11.527478+0000","last_unstale":"2024-05-11T12:21:42.661881+0000","last_undegraded":"2024-05-11T12:21:42.661881+0000","last_fullsized":"2024-05-11T12:21:42.661881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:57:29.001602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665095+0000","last_change":"2024-05-11T12:21:11.526862+0000","last_active":"2024-05-11T12:21:42.665095+0000","last_peered":"2024-05-11T12:21:42.665095+0000","last_clean":"2024-05-11T12:21:42.665095+0000","last_became_active":"2024-05-11T12:21:11.526553+0000","last_became_peered":"2024-05-11T12:21:11.526553+0000","last_unstale":"2024-05-11T12:21:42.665095+0000","last_undegraded":"2024-05-11T12:21:42.665095+0000","last_fullsized":"2024-05-11T12:21:42.665095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:48.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669393+0000","last_change":"2024-05-11T12:21:42.669393+0000","last_active":"2024-05-11T12:21:42.669393+0000","last_peered":"2024-05-11T12:21:42.669393+0000","last_clean":"2024-05-11T12:21:42.669393+0000","last_became_active":"2024-05-11T12:21:42.669000+0000","last_became_peered":"2024-05-11T12:21:42.669000+0000","last_unstale":"2024-05-11T12:21:42.669393+0000","last_undegraded":"2024-05-11T12:21:42.669393+0000","last_fullsized":"2024-05-11T12:21:42.669393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:29.439588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671085+0000","last_change":"2024-05-11T12:21:11.540698+0000","last_active":"2024-05-11T12:21:42.671085+0000","last_peered":"2024-05-11T12:21:42.671085+0000","last_clean":"2024-05-11T12:21:42.671085+0000","last_became_active":"2024-05-11T12:21:11.539899+0000","last_became_peered":"2024-05-11T12:21:11.539899+0000","last_unstale":"2024-05-11T12:21:42.671085+0000","last_undegraded":"2024-05-11T12:21:42.671085+0000","last_fullsized":"2024-05-11T12:21:42.671085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:48.222664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662541+0000","last_change":"2024-05-11T12:21:11.537055+0000","last_active":"2024-05-11T12:21:42.662541+0000","last_peered":"2024-05-11T12:21:42.662541+0000","last_clean":"2024-05-11T12:21:42.662541+0000","last_became_active":"2024-05-11T12:21:11.536625+0000","last_became_peered":"2024-05-11T12:21:11.536625+0000","last_unstale":"2024-05-11T12:21:42.662541+0000","last_undegraded":"2024-05-11T12:21:42.662541+0000","last_fullsized":"2024-05-11T12:21:42.662541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:33.453467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669713+0000","last_change":"2024-05-11T12:21:11.530254+0000","last_active":"2024-05-11T12:21:42.669713+0000","last_peered":"2024-05-11T12:21:42.669713+0000","last_clean":"2024-05-11T12:21:42.669713+0000","last_became_active":"2024-05-11T12:21:11.529890+0000","last_became_peered":"2024-05-11T12:21:11.529890+0000","last_unstale":"2024-05-11T12:21:42.669713+0000","last_undegraded":"2024-05-11T12:21:42.669713+0000","last_fullsized":"2024-05-11T12:21:42.669713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:36.100931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665540+0000","last_change":"2024-05-11T12:21:11.532893+0000","last_active":"2024-05-11T12:21:42.665540+0000","last_peered":"2024-05-11T12:21:42.665540+0000","last_clean":"2024-05-11T12:21:42.665540+0000","last_became_active":"2024-05-11T12:21:11.532539+0000","last_became_peered":"2024-05-11T12:21:11.532539+0000","last_unstale":"2024-05-11T12:21:42.665540+0000","last_undegraded":"2024-05-11T12:21:42.665540+0000","last_fullsized":"2024-05-11T12:21:42.665540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:14:57.888102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669477+0000","last_change":"2024-05-11T12:21:08.510909+0000","last_active":"2024-05-11T12:21:42.669477+0000","last_peered":"2024-05-11T12:21:42.669477+0000","last_clean":"2024-05-11T12:21:42.669477+0000","last_became_active":"2024-05-11T12:21:08.509986+0000","last_became_peered":"2024-05-11T12:21:08.509986+0000","last_unstale":"2024-05-11T12:21:42.669477+0000","last_undegraded":"2024-05-11T12:21:42.669477+0000","last_fullsized":"2024-05-11T12:21:42.669477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:03:36.430087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663971+0000","last_change":"2024-05-11T12:21:11.532403+0000","last_active":"2024-05-11T12:21:42.663971+0000","last_peered":"2024-05-11T12:21:42.663971+0000","last_clean":"2024-05-11T12:21:42.663971+0000","last_became_active":"2024-05-11T12:21:11.531907+0000","last_became_peered":"2024-05-11T12:21:11.531907+0000","last_unstale":"2024-05-11T12:21:42.663971+0000","last_undegraded":"2024-05-11T12:21:42.663971+0000","last_fullsized":"2024-05-11T12:21:42.663971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:53:27.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664274+0000","last_change":"2024-05-11T12:21:08.531664+0000","last_active":"2024-05-11T12:21:42.664274+0000","last_peered":"2024-05-11T12:21:42.664274+0000","last_clean":"2024-05-11T12:21:42.664274+0000","last_became_active":"2024-05-11T12:21:08.531319+0000","last_became_peered":"2024-05-11T12:21:08.531319+0000","last_unstale":"2024-05-11T12:21:42.664274+0000","last_undegraded":"2024-05-11T12:21:42.664274+0000","last_fullsized":"2024-05-11T12:21:42.664274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:16:08.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664533+0000","last_change":"2024-05-11T12:21:11.532132+0000","last_active":"2024-05-11T12:21:42.664533+0000","last_peered":"2024-05-11T12:21:42.664533+0000","last_clean":"2024-05-11T12:21:42.664533+0000","last_became_active":"2024-05-11T12:21:11.529580+0000","last_became_peered":"2024-05-11T12:21:11.529580+0000","last_unstale":"2024-05-11T12:21:42.664533+0000","last_undegraded":"2024-05-11T12:21:42.664533+0000","last_fullsized":"2024-05-11T12:21:42.664533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:29.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.674898+0000","last_change":"2024-05-11T12:21:42.674898+0000","last_active":"2024-05-11T12:21:42.674898+0000","last_peered":"2024-05-11T12:21:42.674898+0000","last_clean":"2024-05-11T12:21:42.674898+0000","last_became_active":"2024-05-11T12:21:42.669853+0000","last_became_peered":"2024-05-11T12:21:42.669853+0000","last_unstale":"2024-05-11T12:21:42.674898+0000","last_undegraded":"2024-05-11T12:21:42.674898+0000","last_fullsized":"2024-05-11T12:21:42.674898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:36.040786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671237+0000","last_change":"2024-05-11T12:21:11.534706+0000","last_active":"2024-05-11T12:21:42.671237+0000","last_peered":"2024-05-11T12:21:42.671237+0000","last_clean":"2024-05-11T12:21:42.671237+0000","last_became_active":"2024-05-11T12:21:11.534301+0000","last_became_peered":"2024-05-11T12:21:11.534301+0000","last_unstale":"2024-05-11T12:21:42.671237+0000","last_undegraded":"2024-05-11T12:21:42.671237+0000","last_fullsized":"2024-05-11T12:21:42.671237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:24.277988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666399+0000","last_change":"2024-05-11T12:21:08.513794+0000","last_active":"2024-05-11T12:21:42.666399+0000","last_peered":"2024-05-11T12:21:42.666399+0000","last_clean":"2024-05-11T12:21:42.666399+0000","last_became_active":"2024-05-11T12:21:08.513383+0000","last_became_peered":"2024-05-11T12:21:08.513383+0000","last_unstale":"2024-05-11T12:21:42.666399+0000","last_undegraded":"2024-05-11T12:21:42.666399+0000","last_fullsized":"2024-05-11T12:21:42.666399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:50.558886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670055+0000","last_change":"2024-05-11T12:21:11.537147+0000","last_active":"2024-05-11T12:21:42.670055+0000","last_peered":"2024-05-11T12:21:42.670055+0000","last_clean":"2024-05-11T12:21:42.670055+0000","last_became_active":"2024-05-11T12:21:11.536739+0000","last_became_peered":"2024-05-11T12:21:11.536739+0000","last_unstale":"2024-05-11T12:21:42.670055+0000","last_undegraded":"2024-05-11T12:21:42.670055+0000","last_fullsized":"2024-05-11T12:21:42.670055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:00:45.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670989+0000","last_change":"2024-05-11T12:21:08.513494+0000","last_active":"2024-05-11T12:21:42.670989+0000","last_peered":"2024-05-11T12:21:42.670989+0000","last_clean":"2024-05-11T12:21:42.670989+0000","last_became_active":"2024-05-11T12:21:08.513089+0000","last_became_peered":"2024-05-11T12:21:08.513089+0000","last_unstale":"2024-05-11T12:21:42.670989+0000","last_undegraded":"2024-05-11T12:21:42.670989+0000","last_fullsized":"2024-05-11T12:21:42.670989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:48:28.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665885+0000","last_change":"2024-05-11T12:21:11.527827+0000","last_active":"2024-05-11T12:21:42.665885+0000","last_peered":"2024-05-11T12:21:42.665885+0000","last_clean":"2024-05-11T12:21:42.665885+0000","last_became_active":"2024-05-11T12:21:11.527397+0000","last_became_peered":"2024-05-11T12:21:11.527397+0000","last_unstale":"2024-05-11T12:21:42.665885+0000","last_undegraded":"2024-05-11T12:21:42.665885+0000","last_fullsized":"2024-05-11T12:21:42.665885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:18.710855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662462+0000","last_change":"2024-05-11T12:21:08.513519+0000","last_active":"2024-05-11T12:21:42.662462+0000","last_peered":"2024-05-11T12:21:42.662462+0000","last_clean":"2024-05-11T12:21:42.662462+0000","last_became_active":"2024-05-11T12:21:08.512566+0000","last_became_peered":"2024-05-11T12:21:08.512566+0000","last_unstale":"2024-05-11T12:21:42.662462+0000","last_undegraded":"2024-05-11T12:21:42.662462+0000","last_fullsized":"2024-05-11T12:21:42.662462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:56.104043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671126+0000","last_change":"2024-05-11T12:21:11.535364+0000","last_active":"2024-05-11T12:21:42.671126+0000","last_peered":"2024-05-11T12:21:42.671126+0000","last_clean":"2024-05-11T12:21:42.671126+0000","last_became_active":"2024-05-11T12:21:11.534988+0000","last_became_peered":"2024-05-11T12:21:11.534988+0000","last_unstale":"2024-05-11T12:21:42.671126+0000","last_undegraded":"2024-05-11T12:21:42.671126+0000","last_fullsized":"2024-05-11T12:21:42.671126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:25:20.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661686+0000","last_change":"2024-05-11T12:21:08.509913+0000","last_active":"2024-05-11T12:21:42.661686+0000","last_peered":"2024-05-11T12:21:42.661686+0000","last_clean":"2024-05-11T12:21:42.661686+0000","last_became_active":"2024-05-11T12:21:08.509498+0000","last_became_peered":"2024-05-11T12:21:08.509498+0000","last_unstale":"2024-05-11T12:21:42.661686+0000","last_undegraded":"2024-05-11T12:21:42.661686+0000","last_fullsized":"2024-05-11T12:21:42.661686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:43.282700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663416+0000","last_change":"2024-05-11T12:21:11.533108+0000","last_active":"2024-05-11T12:21:42.663416+0000","last_peered":"2024-05-11T12:21:42.663416+0000","last_clean":"2024-05-11T12:21:42.663416+0000","last_became_active":"2024-05-11T12:21:11.532779+0000","last_became_peered":"2024-05-11T12:21:11.532779+0000","last_unstale":"2024-05-11T12:21:42.663416+0000","last_undegraded":"2024-05-11T12:21:42.663416+0000","last_fullsized":"2024-05-11T12:21:42.663416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:55.311786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669903+0000","last_change":"2024-05-11T12:21:08.513355+0000","last_active":"2024-05-11T12:21:42.669903+0000","last_peered":"2024-05-11T12:21:42.669903+0000","last_clean":"2024-05-11T12:21:42.669903+0000","last_became_active":"2024-05-11T12:21:08.512964+0000","last_became_peered":"2024-05-11T12:21:08.512964+0000","last_unstale":"2024-05-11T12:21:42.669903+0000","last_undegraded":"2024-05-11T12:21:42.669903+0000","last_fullsized":"2024-05-11T12:21:42.669903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:12.296865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669837+0000","last_change":"2024-05-11T12:21:11.525958+0000","last_active":"2024-05-11T12:21:42.669837+0000","last_peered":"2024-05-11T12:21:42.669837+0000","last_clean":"2024-05-11T12:21:42.669837+0000","last_became_active":"2024-05-11T12:21:11.525285+0000","last_became_peered":"2024-05-11T12:21:11.525285+0000","last_unstale":"2024-05-11T12:21:42.669837+0000","last_undegraded":"2024-05-11T12:21:42.669837+0000","last_fullsized":"2024-05-11T12:21:42.669837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:24:47.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.673378+0000","last_change":"2024-05-11T12:21:42.673378+0000","last_active":"2024-05-11T12:21:42.673378+0000","last_peered":"2024-05-11T12:21:42.673378+0000","last_clean":"2024-05-11T12:21:42.673378+0000","last_became_active":"2024-05-11T12:21:42.672870+0000","last_became_peered":"2024-05-11T12:21:42.672870+0000","last_unstale":"2024-05-11T12:21:42.673378+0000","last_undegraded":"2024-05-11T12:21:42.673378+0000","last_fullsized":"2024-05-11T12:21:42.673378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:55:41.845162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671927+0000","last_change":"2024-05-11T12:21:11.538946+0000","last_active":"2024-05-11T12:21:42.671927+0000","last_peered":"2024-05-11T12:21:42.671927+0000","last_clean":"2024-05-11T12:21:42.671927+0000","last_became_active":"2024-05-11T12:21:11.535804+0000","last_became_peered":"2024-05-11T12:21:11.535804+0000","last_unstale":"2024-05-11T12:21:42.671927+0000","last_undegraded":"2024-05-11T12:21:42.671927+0000","last_fullsized":"2024-05-11T12:21:42.671927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:37:16.221144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.674203+0000","last_change":"2024-05-11T12:21:42.674203+0000","last_active":"2024-05-11T12:21:42.674203+0000","last_peered":"2024-05-11T12:21:42.674203+0000","last_clean":"2024-05-11T12:21:42.674203+0000","last_became_active":"2024-05-11T12:21:42.673478+0000","last_became_peered":"2024-05-11T12:21:42.673478+0000","last_unstale":"2024-05-11T12:21:42.674203+0000","last_undegraded":"2024-05-11T12:21:42.674203+0000","last_fullsized":"2024-05-11T12:21:42.674203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:20.648528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664568+0000","last_change":"2024-05-11T12:21:11.535346+0000","last_active":"2024-05-11T12:21:42.664568+0000","last_peered":"2024-05-11T12:21:42.664568+0000","last_clean":"2024-05-11T12:21:42.664568+0000","last_became_active":"2024-05-11T12:21:11.530262+0000","last_became_peered":"2024-05-11T12:21:11.530262+0000","last_unstale":"2024-05-11T12:21:42.664568+0000","last_undegraded":"2024-05-11T12:21:42.664568+0000","last_fullsized":"2024-05-11T12:21:42.664568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:13.169797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670177+0000","last_change":"2024-05-11T12:21:08.531598+0000","last_active":"2024-05-11T12:21:42.670177+0000","last_peered":"2024-05-11T12:21:42.670177+0000","last_clean":"2024-05-11T12:21:42.670177+0000","last_became_active":"2024-05-11T12:21:08.531158+0000","last_became_peered":"2024-05-11T12:21:08.531158+0000","last_unstale":"2024-05-11T12:21:42.670177+0000","last_undegraded":"2024-05-11T12:21:42.670177+0000","last_fullsized":"2024-05-11T12:21:42.670177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:43.528151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671482+0000","last_change":"2024-05-11T12:21:11.539001+0000","last_active":"2024-05-11T12:21:42.671482+0000","last_peered":"2024-05-11T12:21:42.671482+0000","last_clean":"2024-05-11T12:21:42.671482+0000","last_became_active":"2024-05-11T12:21:11.537324+0000","last_became_peered":"2024-05-11T12:21:11.537324+0000","last_unstale":"2024-05-11T12:21:42.671482+0000","last_undegraded":"2024-05-11T12:21:42.671482+0000","last_fullsized":"2024-05-11T12:21:42.671482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:03:49.393850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'27","reported_seq":107,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:49.668227+0000","last_change":"2024-05-11T12:21:08.516737+0000","last_active":"2024-05-11T12:21:49.668227+0000","last_peered":"2024-05-11T12:21:49.668227+0000","last_clean":"2024-05-11T12:21:49.668227+0000","last_became_active":"2024-05-11T12:21:08.516316+0000","last_became_peered":"2024-05-11T12:21:08.516316+0000","last_unstale":"2024-05-11T12:21:49.668227+0000","last_undegraded":"2024-05-11T12:21:49.668227+0000","last_fullsized":"2024-05-11T12:21:49.668227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:25.769713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57493,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":54,"num_read_kb":82,"num_write":27,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665479+0000","last_change":"2024-05-11T12:21:11.528553+0000","last_active":"2024-05-11T12:21:42.665479+0000","last_peered":"2024-05-11T12:21:42.665479+0000","last_clean":"2024-05-11T12:21:42.665479+0000","last_became_active":"2024-05-11T12:21:11.528175+0000","last_became_peered":"2024-05-11T12:21:11.528175+0000","last_unstale":"2024-05-11T12:21:42.665479+0000","last_undegraded":"2024-05-11T12:21:42.665479+0000","last_fullsized":"2024-05-11T12:21:42.665479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:46:42.689851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.673360+0000","last_change":"2024-05-11T12:21:42.673360+0000","last_active":"2024-05-11T12:21:42.673360+0000","last_peered":"2024-05-11T12:21:42.673360+0000","last_clean":"2024-05-11T12:21:42.673360+0000","last_became_active":"2024-05-11T12:21:42.664842+0000","last_became_peered":"2024-05-11T12:21:42.664842+0000","last_unstale":"2024-05-11T12:21:42.673360+0000","last_undegraded":"2024-05-11T12:21:42.673360+0000","last_fullsized":"2024-05-11T12:21:42.673360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:40.271658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661675+0000","last_change":"2024-05-11T12:21:11.533157+0000","last_active":"2024-05-11T12:21:42.661675+0000","last_peered":"2024-05-11T12:21:42.661675+0000","last_clean":"2024-05-11T12:21:42.661675+0000","last_became_active":"2024-05-11T12:21:11.532814+0000","last_became_peered":"2024-05-11T12:21:11.532814+0000","last_unstale":"2024-05-11T12:21:42.661675+0000","last_undegraded":"2024-05-11T12:21:42.661675+0000","last_fullsized":"2024-05-11T12:21:42.661675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:30:56.555384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670629+0000","last_change":"2024-05-11T12:21:08.511907+0000","last_active":"2024-05-11T12:21:42.670629+0000","last_peered":"2024-05-11T12:21:42.670629+0000","last_clean":"2024-05-11T12:21:42.670629+0000","last_became_active":"2024-05-11T12:21:08.511296+0000","last_became_peered":"2024-05-11T12:21:08.511296+0000","last_unstale":"2024-05-11T12:21:42.670629+0000","last_undegraded":"2024-05-11T12:21:42.670629+0000","last_fullsized":"2024-05-11T12:21:42.670629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:43:17.374216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666144+0000","last_change":"2024-05-11T12:21:11.537859+0000","last_active":"2024-05-11T12:21:42.666144+0000","last_peered":"2024-05-11T12:21:42.666144+0000","last_clean":"2024-05-11T12:21:42.666144+0000","last_became_active":"2024-05-11T12:21:11.537448+0000","last_became_peered":"2024-05-11T12:21:11.537448+0000","last_unstale":"2024-05-11T12:21:42.666144+0000","last_undegraded":"2024-05-11T12:21:42.666144+0000","last_fullsized":"2024-05-11T12:21:42.666144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:09:33.866539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670748+0000","last_change":"2024-05-11T12:21:08.510228+0000","last_active":"2024-05-11T12:21:42.670748+0000","last_peered":"2024-05-11T12:21:42.670748+0000","last_clean":"2024-05-11T12:21:42.670748+0000","last_became_active":"2024-05-11T12:21:08.509671+0000","last_became_peered":"2024-05-11T12:21:08.509671+0000","last_unstale":"2024-05-11T12:21:42.670748+0000","last_undegraded":"2024-05-11T12:21:42.670748+0000","last_fullsized":"2024-05-11T12:21:42.670748+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:06.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665875+0000","last_change":"2024-05-11T12:21:11.530935+0000","last_active":"2024-05-11T12:21:42.665875+0000","last_peered":"2024-05-11T12:21:42.665875+0000","last_clean":"2024-05-11T12:21:42.665875+0000","last_became_active":"2024-05-11T12:21:11.530579+0000","last_became_peered":"2024-05-11T12:21:11.530579+0000","last_unstale":"2024-05-11T12:21:42.665875+0000","last_undegraded":"2024-05-11T12:21:42.665875+0000","last_fullsized":"2024-05-11T12:21:42.665875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:38.296021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.681114+0000","last_change":"2024-05-11T12:21:42.681114+0000","last_active":"2024-05-11T12:21:42.681114+0000","last_peered":"2024-05-11T12:21:42.681114+0000","last_clean":"2024-05-11T12:21:42.681114+0000","last_became_active":"2024-05-11T12:21:42.666878+0000","last_became_peered":"2024-05-11T12:21:42.666878+0000","last_unstale":"2024-05-11T12:21:42.681114+0000","last_undegraded":"2024-05-11T12:21:42.681114+0000","last_fullsized":"2024-05-11T12:21:42.681114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:45.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670655+0000","last_change":"2024-05-11T12:21:11.537879+0000","last_active":"2024-05-11T12:21:42.670655+0000","last_peered":"2024-05-11T12:21:42.670655+0000","last_clean":"2024-05-11T12:21:42.670655+0000","last_became_active":"2024-05-11T12:21:11.537516+0000","last_became_peered":"2024-05-11T12:21:11.537516+0000","last_unstale":"2024-05-11T12:21:42.670655+0000","last_undegraded":"2024-05-11T12:21:42.670655+0000","last_fullsized":"2024-05-11T12:21:42.670655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:22.302157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666195+0000","last_change":"2024-05-11T12:21:08.510504+0000","last_active":"2024-05-11T12:21:42.666195+0000","last_peered":"2024-05-11T12:21:42.666195+0000","last_clean":"2024-05-11T12:21:42.666195+0000","last_became_active":"2024-05-11T12:21:08.510074+0000","last_became_peered":"2024-05-11T12:21:08.510074+0000","last_unstale":"2024-05-11T12:21:42.666195+0000","last_undegraded":"2024-05-11T12:21:42.666195+0000","last_fullsized":"2024-05-11T12:21:42.666195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:31:49.465252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662223+0000","last_change":"2024-05-11T12:21:11.535680+0000","last_active":"2024-05-11T12:21:42.662223+0000","last_peered":"2024-05-11T12:21:42.662223+0000","last_clean":"2024-05-11T12:21:42.662223+0000","last_became_active":"2024-05-11T12:21:11.535292+0000","last_became_peered":"2024-05-11T12:21:11.535292+0000","last_unstale":"2024-05-11T12:21:42.662223+0000","last_undegraded":"2024-05-11T12:21:42.662223+0000","last_fullsized":"2024-05-11T12:21:42.662223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:57:59.122855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669511+0000","last_change":"2024-05-11T12:21:08.531926+0000","last_active":"2024-05-11T12:21:42.669511+0000","last_peered":"2024-05-11T12:21:42.669511+0000","last_clean":"2024-05-11T12:21:42.669511+0000","last_became_active":"2024-05-11T12:21:08.531548+0000","last_became_peered":"2024-05-11T12:21:08.531548+0000","last_unstale":"2024-05-11T12:21:42.669511+0000","last_undegraded":"2024-05-11T12:21:42.669511+0000","last_fullsized":"2024-05-11T12:21:42.669511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:25:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662360+0000","last_change":"2024-05-11T12:21:11.528595+0000","last_active":"2024-05-11T12:21:42.662360+0000","last_peered":"2024-05-11T12:21:42.662360+0000","last_clean":"2024-05-11T12:21:42.662360+0000","last_became_active":"2024-05-11T12:21:11.527782+0000","last_became_peered":"2024-05-11T12:21:11.527782+0000","last_unstale":"2024-05-11T12:21:42.662360+0000","last_undegraded":"2024-05-11T12:21:42.662360+0000","last_fullsized":"2024-05-11T12:21:42.662360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:31:00.337497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666277+0000","last_change":"2024-05-11T12:21:08.508093+0000","last_active":"2024-05-11T12:21:42.666277+0000","last_peered":"2024-05-11T12:21:42.666277+0000","last_clean":"2024-05-11T12:21:42.666277+0000","last_became_active":"2024-05-11T12:21:08.507710+0000","last_became_peered":"2024-05-11T12:21:08.507710+0000","last_unstale":"2024-05-11T12:21:42.666277+0000","last_undegraded":"2024-05-11T12:21:42.666277+0000","last_fullsized":"2024-05-11T12:21:42.666277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:24.273932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662325+0000","last_change":"2024-05-11T12:21:11.526026+0000","last_active":"2024-05-11T12:21:42.662325+0000","last_peered":"2024-05-11T12:21:42.662325+0000","last_clean":"2024-05-11T12:21:42.662325+0000","last_became_active":"2024-05-11T12:21:11.525628+0000","last_became_peered":"2024-05-11T12:21:11.525628+0000","last_unstale":"2024-05-11T12:21:42.662325+0000","last_undegraded":"2024-05-11T12:21:42.662325+0000","last_fullsized":"2024-05-11T12:21:42.662325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:12.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670218+0000","last_change":"2024-05-11T12:21:11.535808+0000","last_active":"2024-05-11T12:21:42.670218+0000","last_peered":"2024-05-11T12:21:42.670218+0000","last_clean":"2024-05-11T12:21:42.670218+0000","last_became_active":"2024-05-11T12:21:11.535204+0000","last_became_peered":"2024-05-11T12:21:11.535204+0000","last_unstale":"2024-05-11T12:21:42.670218+0000","last_undegraded":"2024-05-11T12:21:42.670218+0000","last_fullsized":"2024-05-11T12:21:42.670218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:36:01.208543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661581+0000","last_change":"2024-05-11T12:21:08.531838+0000","last_active":"2024-05-11T12:21:42.661581+0000","last_peered":"2024-05-11T12:21:42.661581+0000","last_clean":"2024-05-11T12:21:42.661581+0000","last_became_active":"2024-05-11T12:21:08.531491+0000","last_became_peered":"2024-05-11T12:21:08.531491+0000","last_unstale":"2024-05-11T12:21:42.661581+0000","last_undegraded":"2024-05-11T12:21:42.661581+0000","last_fullsized":"2024-05-11T12:21:42.661581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:33.928985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663841+0000","last_change":"2024-05-11T12:21:11.538563+0000","last_active":"2024-05-11T12:21:42.663841+0000","last_peered":"2024-05-11T12:21:42.663841+0000","last_clean":"2024-05-11T12:21:42.663841+0000","last_became_active":"2024-05-11T12:21:11.538200+0000","last_became_peered":"2024-05-11T12:21:11.538200+0000","last_unstale":"2024-05-11T12:21:42.663841+0000","last_undegraded":"2024-05-11T12:21:42.663841+0000","last_fullsized":"2024-05-11T12:21:42.663841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:18.879528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.672698+0000","last_change":"2024-05-11T12:21:08.509972+0000","last_active":"2024-05-11T12:21:42.672698+0000","last_peered":"2024-05-11T12:21:42.672698+0000","last_clean":"2024-05-11T12:21:42.672698+0000","last_became_active":"2024-05-11T12:21:08.509531+0000","last_became_peered":"2024-05-11T12:21:08.509531+0000","last_unstale":"2024-05-11T12:21:42.672698+0000","last_undegraded":"2024-05-11T12:21:42.672698+0000","last_fullsized":"2024-05-11T12:21:42.672698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:47:13.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665982+0000","last_change":"2024-05-11T12:19:42.546860+0000","last_active":"2024-05-11T12:21:42.665982+0000","last_peered":"2024-05-11T12:21:42.665982+0000","last_clean":"2024-05-11T12:21:42.665982+0000","last_became_active":"2024-05-11T12:19:42.541449+0000","last_became_peered":"2024-05-11T12:19:42.541449+0000","last_unstale":"2024-05-11T12:21:42.665982+0000","last_undegraded":"2024-05-11T12:21:42.665982+0000","last_fullsized":"2024-05-11T12:21:42.665982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_clean_scrub_stamp":"2024-05-11T12:16:12.381187+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:45.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661556+0000","last_change":"2024-05-11T12:21:11.536770+0000","last_active":"2024-05-11T12:21:42.661556+0000","last_peered":"2024-05-11T12:21:42.661556+0000","last_clean":"2024-05-11T12:21:42.661556+0000","last_became_active":"2024-05-11T12:21:11.536421+0000","last_became_peered":"2024-05-11T12:21:11.536421+0000","last_unstale":"2024-05-11T12:21:42.661556+0000","last_undegraded":"2024-05-11T12:21:42.661556+0000","last_fullsized":"2024-05-11T12:21:42.661556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:42.217398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.675283+0000","last_change":"2024-05-11T12:21:42.675283+0000","last_active":"2024-05-11T12:21:42.675283+0000","last_peered":"2024-05-11T12:21:42.675283+0000","last_clean":"2024-05-11T12:21:42.675283+0000","last_became_active":"2024-05-11T12:21:42.674775+0000","last_became_peered":"2024-05-11T12:21:42.674775+0000","last_unstale":"2024-05-11T12:21:42.675283+0000","last_undegraded":"2024-05-11T12:21:42.675283+0000","last_fullsized":"2024-05-11T12:21:42.675283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:13:30.143988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665594+0000","last_change":"2024-05-11T12:21:08.533462+0000","last_active":"2024-05-11T12:21:42.665594+0000","last_peered":"2024-05-11T12:21:42.665594+0000","last_clean":"2024-05-11T12:21:42.665594+0000","last_became_active":"2024-05-11T12:21:08.533081+0000","last_became_peered":"2024-05-11T12:21:08.533081+0000","last_unstale":"2024-05-11T12:21:42.665594+0000","last_undegraded":"2024-05-11T12:21:42.665594+0000","last_fullsized":"2024-05-11T12:21:42.665594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:01:48.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661904+0000","last_change":"2024-05-11T12:21:11.539484+0000","last_active":"2024-05-11T12:21:42.661904+0000","last_peered":"2024-05-11T12:21:42.661904+0000","last_clean":"2024-05-11T12:21:42.661904+0000","last_became_active":"2024-05-11T12:21:11.536810+0000","last_became_peered":"2024-05-11T12:21:11.536810+0000","last_unstale":"2024-05-11T12:21:42.661904+0000","last_undegraded":"2024-05-11T12:21:42.661904+0000","last_fullsized":"2024-05-11T12:21:42.661904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:47:41.461766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:44.602235+0000","last_change":"2024-05-11T12:21:08.510519+0000","last_active":"2024-05-11T12:21:44.602235+0000","last_peered":"2024-05-11T12:21:44.602235+0000","last_clean":"2024-05-11T12:21:44.602235+0000","last_became_active":"2024-05-11T12:21:08.510180+0000","last_became_peered":"2024-05-11T12:21:08.510180+0000","last_unstale":"2024-05-11T12:21:44.602235+0000","last_undegraded":"2024-05-11T12:21:44.602235+0000","last_fullsized":"2024-05-11T12:21:44.602235+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:24.537043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664175+0000","last_change":"2024-05-11T12:21:11.533726+0000","last_active":"2024-05-11T12:21:42.664175+0000","last_peered":"2024-05-11T12:21:42.664175+0000","last_clean":"2024-05-11T12:21:42.664175+0000","last_became_active":"2024-05-11T12:21:11.533393+0000","last_became_peered":"2024-05-11T12:21:11.533393+0000","last_unstale":"2024-05-11T12:21:42.664175+0000","last_undegraded":"2024-05-11T12:21:42.664175+0000","last_fullsized":"2024-05-11T12:21:42.664175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:03.071666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670305+0000","last_change":"2024-05-11T12:21:09.036622+0000","last_active":"2024-05-11T12:21:42.670305+0000","last_peered":"2024-05-11T12:21:42.670305+0000","last_clean":"2024-05-11T12:21:42.670305+0000","last_became_active":"2024-05-11T12:21:09.036209+0000","last_became_peered":"2024-05-11T12:21:09.036209+0000","last_unstale":"2024-05-11T12:21:42.670305+0000","last_undegraded":"2024-05-11T12:21:42.670305+0000","last_fullsized":"2024-05-11T12:21:42.670305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:29:58.947588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665811+0000","last_change":"2024-05-11T12:21:11.531751+0000","last_active":"2024-05-11T12:21:42.665811+0000","last_peered":"2024-05-11T12:21:42.665811+0000","last_clean":"2024-05-11T12:21:42.665811+0000","last_became_active":"2024-05-11T12:21:11.531071+0000","last_became_peered":"2024-05-11T12:21:11.531071+0000","last_unstale":"2024-05-11T12:21:42.665811+0000","last_undegraded":"2024-05-11T12:21:42.665811+0000","last_fullsized":"2024-05-11T12:21:42.665811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:48.610370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671290+0000","last_change":"2024-05-11T12:21:08.533435+0000","last_active":"2024-05-11T12:21:42.671290+0000","last_peered":"2024-05-11T12:21:42.671290+0000","last_clean":"2024-05-11T12:21:42.671290+0000","last_became_active":"2024-05-11T12:21:08.533100+0000","last_became_peered":"2024-05-11T12:21:08.533100+0000","last_unstale":"2024-05-11T12:21:42.671290+0000","last_undegraded":"2024-05-11T12:21:42.671290+0000","last_fullsized":"2024-05-11T12:21:42.671290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:17.619015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.667054+0000","last_change":"2024-05-11T12:21:11.526140+0000","last_active":"2024-05-11T12:21:42.667054+0000","last_peered":"2024-05-11T12:21:42.667054+0000","last_clean":"2024-05-11T12:21:42.667054+0000","last_became_active":"2024-05-11T12:21:11.525810+0000","last_became_peered":"2024-05-11T12:21:11.525810+0000","last_unstale":"2024-05-11T12:21:42.667054+0000","last_undegraded":"2024-05-11T12:21:42.667054+0000","last_fullsized":"2024-05-11T12:21:42.667054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:14:04.160379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664274+0000","last_change":"2024-05-11T12:21:08.508031+0000","last_active":"2024-05-11T12:21:42.664274+0000","last_peered":"2024-05-11T12:21:42.664274+0000","last_clean":"2024-05-11T12:21:42.664274+0000","last_became_active":"2024-05-11T12:21:08.507703+0000","last_became_peered":"2024-05-11T12:21:08.507703+0000","last_unstale":"2024-05-11T12:21:42.664274+0000","last_undegraded":"2024-05-11T12:21:42.664274+0000","last_fullsized":"2024-05-11T12:21:42.664274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:05.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661987+0000","last_change":"2024-05-11T12:21:11.528293+0000","last_active":"2024-05-11T12:21:42.661987+0000","last_peered":"2024-05-11T12:21:42.661987+0000","last_clean":"2024-05-11T12:21:42.661987+0000","last_became_active":"2024-05-11T12:21:11.527862+0000","last_became_peered":"2024-05-11T12:21:11.527862+0000","last_unstale":"2024-05-11T12:21:42.661987+0000","last_undegraded":"2024-05-11T12:21:42.661987+0000","last_fullsized":"2024-05-11T12:21:42.661987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:07.335829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665126+0000","last_change":"2024-05-11T12:21:08.510138+0000","last_active":"2024-05-11T12:21:42.665126+0000","last_peered":"2024-05-11T12:21:42.665126+0000","last_clean":"2024-05-11T12:21:42.665126+0000","last_became_active":"2024-05-11T12:21:08.509566+0000","last_became_peered":"2024-05-11T12:21:08.509566+0000","last_unstale":"2024-05-11T12:21:42.665126+0000","last_undegraded":"2024-05-11T12:21:42.665126+0000","last_fullsized":"2024-05-11T12:21:42.665126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:24:58.174305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663889+0000","last_change":"2024-05-11T12:21:11.528730+0000","last_active":"2024-05-11T12:21:42.663889+0000","last_peered":"2024-05-11T12:21:42.663889+0000","last_clean":"2024-05-11T12:21:42.663889+0000","last_became_active":"2024-05-11T12:21:11.528355+0000","last_became_peered":"2024-05-11T12:21:11.528355+0000","last_unstale":"2024-05-11T12:21:42.663889+0000","last_undegraded":"2024-05-11T12:21:42.663889+0000","last_fullsized":"2024-05-11T12:21:42.663889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:49:07.559329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'2","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.683706+0000","last_change":"2024-05-11T12:21:42.683706+0000","last_active":"2024-05-11T12:21:42.683706+0000","last_peered":"2024-05-11T12:21:42.683706+0000","last_clean":"2024-05-11T12:21:42.683706+0000","last_became_active":"2024-05-11T12:21:42.675791+0000","last_became_peered":"2024-05-11T12:21:42.675791+0000","last_unstale":"2024-05-11T12:21:42.683706+0000","last_undegraded":"2024-05-11T12:21:42.683706+0000","last_fullsized":"2024-05-11T12:21:42.683706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:13:26.737525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4],"acting":[8,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666251+0000","last_change":"2024-05-11T12:21:11.544153+0000","last_active":"2024-05-11T12:21:42.666251+0000","last_peered":"2024-05-11T12:21:42.666251+0000","last_clean":"2024-05-11T12:21:42.666251+0000","last_became_active":"2024-05-11T12:21:11.543772+0000","last_became_peered":"2024-05-11T12:21:11.543772+0000","last_unstale":"2024-05-11T12:21:42.666251+0000","last_undegraded":"2024-05-11T12:21:42.666251+0000","last_fullsized":"2024-05-11T12:21:42.666251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:54:30.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665919+0000","last_change":"2024-05-11T12:21:09.036409+0000","last_active":"2024-05-11T12:21:42.665919+0000","last_peered":"2024-05-11T12:21:42.665919+0000","last_clean":"2024-05-11T12:21:42.665919+0000","last_became_active":"2024-05-11T12:21:09.036009+0000","last_became_peered":"2024-05-11T12:21:09.036009+0000","last_unstale":"2024-05-11T12:21:42.665919+0000","last_undegraded":"2024-05-11T12:21:42.665919+0000","last_fullsized":"2024-05-11T12:21:42.665919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:55:28.736718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666612+0000","last_change":"2024-05-11T12:21:11.542685+0000","last_active":"2024-05-11T12:21:42.666612+0000","last_peered":"2024-05-11T12:21:42.666612+0000","last_clean":"2024-05-11T12:21:42.666612+0000","last_became_active":"2024-05-11T12:21:11.542130+0000","last_became_peered":"2024-05-11T12:21:11.542130+0000","last_unstale":"2024-05-11T12:21:42.666612+0000","last_undegraded":"2024-05-11T12:21:42.666612+0000","last_fullsized":"2024-05-11T12:21:42.666612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:22.578809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669920+0000","last_change":"2024-05-11T12:21:08.510869+0000","last_active":"2024-05-11T12:21:42.669920+0000","last_peered":"2024-05-11T12:21:42.669920+0000","last_clean":"2024-05-11T12:21:42.669920+0000","last_became_active":"2024-05-11T12:21:08.510510+0000","last_became_peered":"2024-05-11T12:21:08.510510+0000","last_unstale":"2024-05-11T12:21:42.669920+0000","last_undegraded":"2024-05-11T12:21:42.669920+0000","last_fullsized":"2024-05-11T12:21:42.669920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:50.002313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666183+0000","last_change":"2024-05-11T12:21:11.538622+0000","last_active":"2024-05-11T12:21:42.666183+0000","last_peered":"2024-05-11T12:21:42.666183+0000","last_clean":"2024-05-11T12:21:42.666183+0000","last_became_active":"2024-05-11T12:21:11.538229+0000","last_became_peered":"2024-05-11T12:21:11.538229+0000","last_unstale":"2024-05-11T12:21:42.666183+0000","last_undegraded":"2024-05-11T12:21:42.666183+0000","last_fullsized":"2024-05-11T12:21:42.666183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:01:15.912428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.677421+0000","last_change":"2024-05-11T12:21:42.677421+0000","last_active":"2024-05-11T12:21:42.677421+0000","last_peered":"2024-05-11T12:21:42.677421+0000","last_clean":"2024-05-11T12:21:42.677421+0000","last_became_active":"2024-05-11T12:21:42.672695+0000","last_became_peered":"2024-05-11T12:21:42.672695+0000","last_unstale":"2024-05-11T12:21:42.677421+0000","last_undegraded":"2024-05-11T12:21:42.677421+0000","last_fullsized":"2024-05-11T12:21:42.677421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:30:27.606816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666671+0000","last_change":"2024-05-11T12:21:11.536149+0000","last_active":"2024-05-11T12:21:42.666671+0000","last_peered":"2024-05-11T12:21:42.666671+0000","last_clean":"2024-05-11T12:21:42.666671+0000","last_became_active":"2024-05-11T12:21:11.535763+0000","last_became_peered":"2024-05-11T12:21:11.535763+0000","last_unstale":"2024-05-11T12:21:42.666671+0000","last_undegraded":"2024-05-11T12:21:42.666671+0000","last_fullsized":"2024-05-11T12:21:42.666671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:46:15.246610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662148+0000","last_change":"2024-05-11T12:21:11.534318+0000","last_active":"2024-05-11T12:21:42.662148+0000","last_peered":"2024-05-11T12:21:42.662148+0000","last_clean":"2024-05-11T12:21:42.662148+0000","last_became_active":"2024-05-11T12:21:11.533949+0000","last_became_peered":"2024-05-11T12:21:11.533949+0000","last_unstale":"2024-05-11T12:21:42.662148+0000","last_undegraded":"2024-05-11T12:21:42.662148+0000","last_fullsized":"2024-05-11T12:21:42.662148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:41:42.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665121+0000","last_change":"2024-05-11T12:21:11.529276+0000","last_active":"2024-05-11T12:21:42.665121+0000","last_peered":"2024-05-11T12:21:42.665121+0000","last_clean":"2024-05-11T12:21:42.665121+0000","last_became_active":"2024-05-11T12:21:11.528857+0000","last_became_peered":"2024-05-11T12:21:11.528857+0000","last_unstale":"2024-05-11T12:21:42.665121+0000","last_undegraded":"2024-05-11T12:21:42.665121+0000","last_fullsized":"2024-05-11T12:21:42.665121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:54.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664991+0000","last_change":"2024-05-11T12:21:11.532342+0000","last_active":"2024-05-11T12:21:42.664991+0000","last_peered":"2024-05-11T12:21:42.664991+0000","last_clean":"2024-05-11T12:21:42.664991+0000","last_became_active":"2024-05-11T12:21:11.531949+0000","last_became_peered":"2024-05-11T12:21:11.531949+0000","last_unstale":"2024-05-11T12:21:42.664991+0000","last_undegraded":"2024-05-11T12:21:42.664991+0000","last_fullsized":"2024-05-11T12:21:42.664991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:23:00.010874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665522+0000","last_change":"2024-05-11T12:21:11.531547+0000","last_active":"2024-05-11T12:21:42.665522+0000","last_peered":"2024-05-11T12:21:42.665522+0000","last_clean":"2024-05-11T12:21:42.665522+0000","last_became_active":"2024-05-11T12:21:11.529702+0000","last_became_peered":"2024-05-11T12:21:11.529702+0000","last_unstale":"2024-05-11T12:21:42.665522+0000","last_undegraded":"2024-05-11T12:21:42.665522+0000","last_fullsized":"2024-05-11T12:21:42.665522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:13.857026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661964+0000","last_change":"2024-05-11T12:21:11.531261+0000","last_active":"2024-05-11T12:21:42.661964+0000","last_peered":"2024-05-11T12:21:42.661964+0000","last_clean":"2024-05-11T12:21:42.661964+0000","last_became_active":"2024-05-11T12:21:11.530561+0000","last_became_peered":"2024-05-11T12:21:11.530561+0000","last_unstale":"2024-05-11T12:21:42.661964+0000","last_undegraded":"2024-05-11T12:21:42.661964+0000","last_fullsized":"2024-05-11T12:21:42.661964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:52:07.654417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662182+0000","last_change":"2024-05-11T12:21:11.533255+0000","last_active":"2024-05-11T12:21:42.662182+0000","last_peered":"2024-05-11T12:21:42.662182+0000","last_clean":"2024-05-11T12:21:42.662182+0000","last_became_active":"2024-05-11T12:21:11.532909+0000","last_became_peered":"2024-05-11T12:21:11.532909+0000","last_unstale":"2024-05-11T12:21:42.662182+0000","last_undegraded":"2024-05-11T12:21:42.662182+0000","last_fullsized":"2024-05-11T12:21:42.662182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:07:39.501482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666539+0000","last_change":"2024-05-11T12:21:11.528856+0000","last_active":"2024-05-11T12:21:42.666539+0000","last_peered":"2024-05-11T12:21:42.666539+0000","last_clean":"2024-05-11T12:21:42.666539+0000","last_became_active":"2024-05-11T12:21:11.528230+0000","last_became_peered":"2024-05-11T12:21:11.528230+0000","last_unstale":"2024-05-11T12:21:42.666539+0000","last_undegraded":"2024-05-11T12:21:42.666539+0000","last_fullsized":"2024-05-11T12:21:42.666539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:40.429045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664005+0000","last_change":"2024-05-11T12:21:11.534606+0000","last_active":"2024-05-11T12:21:42.664005+0000","last_peered":"2024-05-11T12:21:42.664005+0000","last_clean":"2024-05-11T12:21:42.664005+0000","last_became_active":"2024-05-11T12:21:11.534193+0000","last_became_peered":"2024-05-11T12:21:11.534193+0000","last_unstale":"2024-05-11T12:21:42.664005+0000","last_undegraded":"2024-05-11T12:21:42.664005+0000","last_fullsized":"2024-05-11T12:21:42.664005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:00:12.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670022+0000","last_change":"2024-05-11T12:21:11.525959+0000","last_active":"2024-05-11T12:21:42.670022+0000","last_peered":"2024-05-11T12:21:42.670022+0000","last_clean":"2024-05-11T12:21:42.670022+0000","last_became_active":"2024-05-11T12:21:11.525129+0000","last_became_peered":"2024-05-11T12:21:11.525129+0000","last_unstale":"2024-05-11T12:21:42.670022+0000","last_undegraded":"2024-05-11T12:21:42.670022+0000","last_fullsized":"2024-05-11T12:21:42.670022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:28.828336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663994+0000","last_change":"2024-05-11T12:21:11.538591+0000","last_active":"2024-05-11T12:21:42.663994+0000","last_peered":"2024-05-11T12:21:42.663994+0000","last_clean":"2024-05-11T12:21:42.663994+0000","last_became_active":"2024-05-11T12:21:11.537855+0000","last_became_peered":"2024-05-11T12:21:11.537855+0000","last_unstale":"2024-05-11T12:21:42.663994+0000","last_undegraded":"2024-05-11T12:21:42.663994+0000","last_fullsized":"2024-05-11T12:21:42.663994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:52.625763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670959+0000","last_change":"2024-05-11T12:21:11.541185+0000","last_active":"2024-05-11T12:21:42.670959+0000","last_peered":"2024-05-11T12:21:42.670959+0000","last_clean":"2024-05-11T12:21:42.670959+0000","last_became_active":"2024-05-11T12:21:11.540799+0000","last_became_peered":"2024-05-11T12:21:11.540799+0000","last_unstale":"2024-05-11T12:21:42.670959+0000","last_undegraded":"2024-05-11T12:21:42.670959+0000","last_fullsized":"2024-05-11T12:21:42.670959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:24.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661882+0000","last_change":"2024-05-11T12:21:11.527811+0000","last_active":"2024-05-11T12:21:42.661882+0000","last_peered":"2024-05-11T12:21:42.661882+0000","last_clean":"2024-05-11T12:21:42.661882+0000","last_became_active":"2024-05-11T12:21:11.526603+0000","last_became_peered":"2024-05-11T12:21:11.526603+0000","last_unstale":"2024-05-11T12:21:42.661882+0000","last_undegraded":"2024-05-11T12:21:42.661882+0000","last_fullsized":"2024-05-11T12:21:42.661882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:43.833506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662052+0000","last_change":"2024-05-11T12:21:11.530824+0000","last_active":"2024-05-11T12:21:42.662052+0000","last_peered":"2024-05-11T12:21:42.662052+0000","last_clean":"2024-05-11T12:21:42.662052+0000","last_became_active":"2024-05-11T12:21:11.530486+0000","last_became_peered":"2024-05-11T12:21:11.530486+0000","last_unstale":"2024-05-11T12:21:42.662052+0000","last_undegraded":"2024-05-11T12:21:42.662052+0000","last_fullsized":"2024-05-11T12:21:42.662052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:47.054822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247249,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58291,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":60,"num_read_kb":88,"num_write":82,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":170063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678039,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":427807,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874268,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":1075945,"data_compressed":41556,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037794,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":872448,"data_stored":428355,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234022,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":428377,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":1018840,"data_compressed":41556,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":872448,"data_stored":427890,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":929792,"data_stored":480456,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":427863,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":485266,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":428111,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":427795,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1018440,"data_compressed":41556,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:21:53.997 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T12:21:54.252 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:54.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:54 smithi050 ceph-mon[34072]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:21:54.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:54 smithi050 ceph-mon[34072]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.3 KiB/s wr, 4 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:54 smithi047 ceph-mon[30375]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:21:55.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:54 smithi047 ceph-mon[30375]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.3 KiB/s wr, 4 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:54 smithi064 ceph-mon[34537]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:21:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:54 smithi064 ceph-mon[34537]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.3 KiB/s wr, 4 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.813+0000 7f55ac480700 1 -- 172.21.15.47:0/456811700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41024e0 msgr2=0x7f55a4102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.813+0000 7f55ac480700 1 --2- 172.21.15.47:0/456811700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41024e0 0x7f55a4102960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5594009990 tx=0x7f5594009ca0 comp rx=0 tx=0).stop 2024-05-11T12:21:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 -- 172.21.15.47:0/456811700 shutdown_connections 2024-05-11T12:21:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 --2- 172.21.15.47:0/456811700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55a4102ea0 0x7f55a410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:55.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 --2- 172.21.15.47:0/456811700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55a41084e0 0x7f55a41088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 --2- 172.21.15.47:0/456811700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41024e0 0x7f55a4102960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 -- 172.21.15.47:0/456811700 >> 172.21.15.47:0/456811700 conn(0x7f55a40784a0 msgr2=0x7f55a40788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 -- 172.21.15.47:0/456811700 shutdown_connections 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 -- 172.21.15.47:0/456811700 wait complete. 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 Processor -- start 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 -- start start 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.814+0000 7f55ac480700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55a41024e0 0x7f55a419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55ac480700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55a4102ea0 0x7f55a419fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55ac480700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41084e0 0x7f55a4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55ac480700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f55a4112670 con 0x7f55a41084e0 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55ac480700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41124f0 con 0x7f55a4102ea0 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55ac480700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f55a41127f0 con 0x7f55a41024e0 2024-05-11T12:21:55.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41084e0 0x7f55a4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41084e0 0x7f55a4199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45610/0 (socket says 172.21.15.47:45610) 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 -- 172.21.15.47:0/4258318876 learned_addr learned my addr 172.21.15.47:0/4258318876 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55a9a1b700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55a4102ea0 0x7f55a419fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aa21c700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55a41024e0 0x7f55a419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 -- 172.21.15.47:0/4258318876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55a41024e0 msgr2=0x7f55a419f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55a41024e0 0x7f55a419f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 -- 172.21.15.47:0/4258318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55a4102ea0 msgr2=0x7f55a419fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55a4102ea0 0x7f55a419fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 -- 172.21.15.47:0/4258318876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a419a1b0 con 0x7f55a41084e0 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55a9a1b700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55a4102ea0 0x7f55a419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aa21c700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55a41024e0 0x7f55a419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55aaa1d700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41084e0 0x7f55a4199a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f559c00c900 tx=0x7f559c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f559b7fe700 1 -- 172.21.15.47:0/4258318876 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c007b00 con 0x7f55a41084e0 2024-05-11T12:21:55.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f559b7fe700 1 -- 172.21.15.47:0/4258318876 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c007ca0 con 0x7f55a41084e0 2024-05-11T12:21:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a419a4a0 con 0x7f55a41084e0 2024-05-11T12:21:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f559b7fe700 1 -- 172.21.15.47:0/4258318876 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c018660 con 0x7f55a41084e0 2024-05-11T12:21:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.815+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a41a6760 con 0x7f55a41084e0 2024-05-11T12:21:55.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.816+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a406a240 con 0x7f55a41084e0 2024-05-11T12:21:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.818+0000 7f559b7fe700 1 -- 172.21.15.47:0/4258318876 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f559c00f450 con 0x7f55a41084e0 2024-05-11T12:21:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.818+0000 7f559b7fe700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55900b2990 0x7f55900b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.819+0000 7f55aa21c700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55900b2990 0x7f55900b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.819+0000 7f559b7fe700 1 -- 172.21.15.47:0/4258318876 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f559c0e2300 con 0x7f55a41084e0 2024-05-11T12:21:55.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.819+0000 7f55aa21c700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55900b2990 0x7f55900b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55a419ab90 tx=0x7f55a00073e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:55.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:55.822+0000 7f559b7fe700 1 -- 172.21.15.47:0/4258318876 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f559c014030 con 0x7f55a41084e0 2024-05-11T12:21:56.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.033+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f55a4063990 con 0x7f55900b2990 2024-05-11T12:21:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.040+0000 7f559b7fe700 1 -- 172.21.15.47:0/4258318876 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258878 (secure 0 0 0) 0x7f55a4063990 con 0x7f55900b2990 2024-05-11T12:21:56.045 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55900b2990 msgr2=0x7f55900b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55900b2990 0x7f55900b4e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f55a419ab90 tx=0x7f55a00073e0 comp rx=0 tx=0).stop 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41084e0 msgr2=0x7f55a4199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41084e0 0x7f55a4199a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f559c00c900 tx=0x7f559c00ccc0 comp rx=0 tx=0).stop 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 shutdown_connections 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f55a41084e0 0x7f55a4199a50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f55a41024e0 0x7f55a419f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55900b2990 0x7f55900b4e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 --2- 172.21.15.47:0/4258318876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f55a4102ea0 0x7f55a419fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 >> 172.21.15.47:0/4258318876 conn(0x7f55a40784a0 msgr2=0x7f55a406c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 shutdown_connections 2024-05-11T12:21:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:56.045+0000 7f55ac480700 1 -- 172.21.15.47:0/4258318876 wait complete. 2024-05-11T12:21:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-05-11T12:21:56.735 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-05-11T12:21:55.541991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653150,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":147,"num_read_kb":162,"num_write":210,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951874,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393072,"kb_used_data":10600,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124548752,"statfs":{"total":1151940427776,"available":1151537922048,"internally_reserved":0,"allocated":10854400,"data_stored":7084212,"data_compressed":176607,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33784,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":-6,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":50,"num_write":25,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002987"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662405+0000","last_change":"2024-05-11T12:21:11.536657+0000","last_active":"2024-05-11T12:21:42.662405+0000","last_peered":"2024-05-11T12:21:42.662405+0000","last_clean":"2024-05-11T12:21:42.662405+0000","last_became_active":"2024-05-11T12:21:11.536243+0000","last_became_peered":"2024-05-11T12:21:11.536243+0000","last_unstale":"2024-05-11T12:21:42.662405+0000","last_undegraded":"2024-05-11T12:21:42.662405+0000","last_fullsized":"2024-05-11T12:21:42.662405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:29:20.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665724+0000","last_change":"2024-05-11T12:21:11.527602+0000","last_active":"2024-05-11T12:21:42.665724+0000","last_peered":"2024-05-11T12:21:42.665724+0000","last_clean":"2024-05-11T12:21:42.665724+0000","last_became_active":"2024-05-11T12:21:11.527309+0000","last_became_peered":"2024-05-11T12:21:11.527309+0000","last_unstale":"2024-05-11T12:21:42.665724+0000","last_undegraded":"2024-05-11T12:21:42.665724+0000","last_fullsized":"2024-05-11T12:21:42.665724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:03.764570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665570+0000","last_change":"2024-05-11T12:21:11.529094+0000","last_active":"2024-05-11T12:21:42.665570+0000","last_peered":"2024-05-11T12:21:42.665570+0000","last_clean":"2024-05-11T12:21:42.665570+0000","last_became_active":"2024-05-11T12:21:11.528796+0000","last_became_peered":"2024-05-11T12:21:11.528796+0000","last_unstale":"2024-05-11T12:21:42.665570+0000","last_undegraded":"2024-05-11T12:21:42.665570+0000","last_fullsized":"2024-05-11T12:21:42.665570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:46:41.098066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669710+0000","last_change":"2024-05-11T12:21:11.528903+0000","last_active":"2024-05-11T12:21:42.669710+0000","last_peered":"2024-05-11T12:21:42.669710+0000","last_clean":"2024-05-11T12:21:42.669710+0000","last_became_active":"2024-05-11T12:21:11.528545+0000","last_became_peered":"2024-05-11T12:21:11.528545+0000","last_unstale":"2024-05-11T12:21:42.669710+0000","last_undegraded":"2024-05-11T12:21:42.669710+0000","last_fullsized":"2024-05-11T12:21:42.669710+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:56.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665264+0000","last_change":"2024-05-11T12:21:11.521074+0000","last_active":"2024-05-11T12:21:42.665264+0000","last_peered":"2024-05-11T12:21:42.665264+0000","last_clean":"2024-05-11T12:21:42.665264+0000","last_became_active":"2024-05-11T12:21:11.520675+0000","last_became_peered":"2024-05-11T12:21:11.520675+0000","last_unstale":"2024-05-11T12:21:42.665264+0000","last_undegraded":"2024-05-11T12:21:42.665264+0000","last_fullsized":"2024-05-11T12:21:42.665264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:22:50.952687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671243+0000","last_change":"2024-05-11T12:21:42.671243+0000","last_active":"2024-05-11T12:21:42.671243+0000","last_peered":"2024-05-11T12:21:42.671243+0000","last_clean":"2024-05-11T12:21:42.671243+0000","last_became_active":"2024-05-11T12:21:42.670801+0000","last_became_peered":"2024-05-11T12:21:42.670801+0000","last_unstale":"2024-05-11T12:21:42.671243+0000","last_undegraded":"2024-05-11T12:21:42.671243+0000","last_fullsized":"2024-05-11T12:21:42.671243+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:02.539855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665943+0000","last_change":"2024-05-11T12:21:11.529961+0000","last_active":"2024-05-11T12:21:42.665943+0000","last_peered":"2024-05-11T12:21:42.665943+0000","last_clean":"2024-05-11T12:21:42.665943+0000","last_became_active":"2024-05-11T12:21:11.529412+0000","last_became_peered":"2024-05-11T12:21:11.529412+0000","last_unstale":"2024-05-11T12:21:42.665943+0000","last_undegraded":"2024-05-11T12:21:42.665943+0000","last_fullsized":"2024-05-11T12:21:42.665943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:00.579881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663513+0000","last_change":"2024-05-11T12:21:11.532874+0000","last_active":"2024-05-11T12:21:42.663513+0000","last_peered":"2024-05-11T12:21:42.663513+0000","last_clean":"2024-05-11T12:21:42.663513+0000","last_became_active":"2024-05-11T12:21:11.532237+0000","last_became_peered":"2024-05-11T12:21:11.532237+0000","last_unstale":"2024-05-11T12:21:42.663513+0000","last_undegraded":"2024-05-11T12:21:42.663513+0000","last_fullsized":"2024-05-11T12:21:42.663513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:13.770719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670901+0000","last_change":"2024-05-11T12:21:11.536465+0000","last_active":"2024-05-11T12:21:42.670901+0000","last_peered":"2024-05-11T12:21:42.670901+0000","last_clean":"2024-05-11T12:21:42.670901+0000","last_became_active":"2024-05-11T12:21:11.536057+0000","last_became_peered":"2024-05-11T12:21:11.536057+0000","last_unstale":"2024-05-11T12:21:42.670901+0000","last_undegraded":"2024-05-11T12:21:42.670901+0000","last_fullsized":"2024-05-11T12:21:42.670901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:22:45.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662835+0000","last_change":"2024-05-11T12:21:11.528187+0000","last_active":"2024-05-11T12:21:42.662835+0000","last_peered":"2024-05-11T12:21:42.662835+0000","last_clean":"2024-05-11T12:21:42.662835+0000","last_became_active":"2024-05-11T12:21:11.527780+0000","last_became_peered":"2024-05-11T12:21:11.527780+0000","last_unstale":"2024-05-11T12:21:42.662835+0000","last_undegraded":"2024-05-11T12:21:42.662835+0000","last_fullsized":"2024-05-11T12:21:42.662835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:23.484372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666391+0000","last_change":"2024-05-11T12:21:11.543388+0000","last_active":"2024-05-11T12:21:42.666391+0000","last_peered":"2024-05-11T12:21:42.666391+0000","last_clean":"2024-05-11T12:21:42.666391+0000","last_became_active":"2024-05-11T12:21:11.542984+0000","last_became_peered":"2024-05-11T12:21:11.542984+0000","last_unstale":"2024-05-11T12:21:42.666391+0000","last_undegraded":"2024-05-11T12:21:42.666391+0000","last_fullsized":"2024-05-11T12:21:42.666391+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:52.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.672467+0000","last_change":"2024-05-11T12:21:11.523355+0000","last_active":"2024-05-11T12:21:42.672467+0000","last_peered":"2024-05-11T12:21:42.672467+0000","last_clean":"2024-05-11T12:21:42.672467+0000","last_became_active":"2024-05-11T12:21:11.523007+0000","last_became_peered":"2024-05-11T12:21:11.523007+0000","last_unstale":"2024-05-11T12:21:42.672467+0000","last_undegraded":"2024-05-11T12:21:42.672467+0000","last_fullsized":"2024-05-11T12:21:42.672467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:57.197028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661881+0000","last_change":"2024-05-11T12:21:11.528072+0000","last_active":"2024-05-11T12:21:42.661881+0000","last_peered":"2024-05-11T12:21:42.661881+0000","last_clean":"2024-05-11T12:21:42.661881+0000","last_became_active":"2024-05-11T12:21:11.527478+0000","last_became_peered":"2024-05-11T12:21:11.527478+0000","last_unstale":"2024-05-11T12:21:42.661881+0000","last_undegraded":"2024-05-11T12:21:42.661881+0000","last_fullsized":"2024-05-11T12:21:42.661881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:57:29.001602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665095+0000","last_change":"2024-05-11T12:21:11.526862+0000","last_active":"2024-05-11T12:21:42.665095+0000","last_peered":"2024-05-11T12:21:42.665095+0000","last_clean":"2024-05-11T12:21:42.665095+0000","last_became_active":"2024-05-11T12:21:11.526553+0000","last_became_peered":"2024-05-11T12:21:11.526553+0000","last_unstale":"2024-05-11T12:21:42.665095+0000","last_undegraded":"2024-05-11T12:21:42.665095+0000","last_fullsized":"2024-05-11T12:21:42.665095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:48.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669393+0000","last_change":"2024-05-11T12:21:42.669393+0000","last_active":"2024-05-11T12:21:42.669393+0000","last_peered":"2024-05-11T12:21:42.669393+0000","last_clean":"2024-05-11T12:21:42.669393+0000","last_became_active":"2024-05-11T12:21:42.669000+0000","last_became_peered":"2024-05-11T12:21:42.669000+0000","last_unstale":"2024-05-11T12:21:42.669393+0000","last_undegraded":"2024-05-11T12:21:42.669393+0000","last_fullsized":"2024-05-11T12:21:42.669393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:29.439588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671085+0000","last_change":"2024-05-11T12:21:11.540698+0000","last_active":"2024-05-11T12:21:42.671085+0000","last_peered":"2024-05-11T12:21:42.671085+0000","last_clean":"2024-05-11T12:21:42.671085+0000","last_became_active":"2024-05-11T12:21:11.539899+0000","last_became_peered":"2024-05-11T12:21:11.539899+0000","last_unstale":"2024-05-11T12:21:42.671085+0000","last_undegraded":"2024-05-11T12:21:42.671085+0000","last_fullsized":"2024-05-11T12:21:42.671085+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:48.222664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662541+0000","last_change":"2024-05-11T12:21:11.537055+0000","last_active":"2024-05-11T12:21:42.662541+0000","last_peered":"2024-05-11T12:21:42.662541+0000","last_clean":"2024-05-11T12:21:42.662541+0000","last_became_active":"2024-05-11T12:21:11.536625+0000","last_became_peered":"2024-05-11T12:21:11.536625+0000","last_unstale":"2024-05-11T12:21:42.662541+0000","last_undegraded":"2024-05-11T12:21:42.662541+0000","last_fullsized":"2024-05-11T12:21:42.662541+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:33.453467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669713+0000","last_change":"2024-05-11T12:21:11.530254+0000","last_active":"2024-05-11T12:21:42.669713+0000","last_peered":"2024-05-11T12:21:42.669713+0000","last_clean":"2024-05-11T12:21:42.669713+0000","last_became_active":"2024-05-11T12:21:11.529890+0000","last_became_peered":"2024-05-11T12:21:11.529890+0000","last_unstale":"2024-05-11T12:21:42.669713+0000","last_undegraded":"2024-05-11T12:21:42.669713+0000","last_fullsized":"2024-05-11T12:21:42.669713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:36.100931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665540+0000","last_change":"2024-05-11T12:21:11.532893+0000","last_active":"2024-05-11T12:21:42.665540+0000","last_peered":"2024-05-11T12:21:42.665540+0000","last_clean":"2024-05-11T12:21:42.665540+0000","last_became_active":"2024-05-11T12:21:11.532539+0000","last_became_peered":"2024-05-11T12:21:11.532539+0000","last_unstale":"2024-05-11T12:21:42.665540+0000","last_undegraded":"2024-05-11T12:21:42.665540+0000","last_fullsized":"2024-05-11T12:21:42.665540+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:14:57.888102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'4","reported_seq":31,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669477+0000","last_change":"2024-05-11T12:21:08.510909+0000","last_active":"2024-05-11T12:21:42.669477+0000","last_peered":"2024-05-11T12:21:42.669477+0000","last_clean":"2024-05-11T12:21:42.669477+0000","last_became_active":"2024-05-11T12:21:08.509986+0000","last_became_peered":"2024-05-11T12:21:08.509986+0000","last_unstale":"2024-05-11T12:21:42.669477+0000","last_undegraded":"2024-05-11T12:21:42.669477+0000","last_fullsized":"2024-05-11T12:21:42.669477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:03:36.430087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663971+0000","last_change":"2024-05-11T12:21:11.532403+0000","last_active":"2024-05-11T12:21:42.663971+0000","last_peered":"2024-05-11T12:21:42.663971+0000","last_clean":"2024-05-11T12:21:42.663971+0000","last_became_active":"2024-05-11T12:21:11.531907+0000","last_became_peered":"2024-05-11T12:21:11.531907+0000","last_unstale":"2024-05-11T12:21:42.663971+0000","last_undegraded":"2024-05-11T12:21:42.663971+0000","last_fullsized":"2024-05-11T12:21:42.663971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:53:27.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664274+0000","last_change":"2024-05-11T12:21:08.531664+0000","last_active":"2024-05-11T12:21:42.664274+0000","last_peered":"2024-05-11T12:21:42.664274+0000","last_clean":"2024-05-11T12:21:42.664274+0000","last_became_active":"2024-05-11T12:21:08.531319+0000","last_became_peered":"2024-05-11T12:21:08.531319+0000","last_unstale":"2024-05-11T12:21:42.664274+0000","last_undegraded":"2024-05-11T12:21:42.664274+0000","last_fullsized":"2024-05-11T12:21:42.664274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:16:08.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664533+0000","last_change":"2024-05-11T12:21:11.532132+0000","last_active":"2024-05-11T12:21:42.664533+0000","last_peered":"2024-05-11T12:21:42.664533+0000","last_clean":"2024-05-11T12:21:42.664533+0000","last_became_active":"2024-05-11T12:21:11.529580+0000","last_became_peered":"2024-05-11T12:21:11.529580+0000","last_unstale":"2024-05-11T12:21:42.664533+0000","last_undegraded":"2024-05-11T12:21:42.664533+0000","last_fullsized":"2024-05-11T12:21:42.664533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:29.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.674898+0000","last_change":"2024-05-11T12:21:42.674898+0000","last_active":"2024-05-11T12:21:42.674898+0000","last_peered":"2024-05-11T12:21:42.674898+0000","last_clean":"2024-05-11T12:21:42.674898+0000","last_became_active":"2024-05-11T12:21:42.669853+0000","last_became_peered":"2024-05-11T12:21:42.669853+0000","last_unstale":"2024-05-11T12:21:42.674898+0000","last_undegraded":"2024-05-11T12:21:42.674898+0000","last_fullsized":"2024-05-11T12:21:42.674898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:36.040786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671237+0000","last_change":"2024-05-11T12:21:11.534706+0000","last_active":"2024-05-11T12:21:42.671237+0000","last_peered":"2024-05-11T12:21:42.671237+0000","last_clean":"2024-05-11T12:21:42.671237+0000","last_became_active":"2024-05-11T12:21:11.534301+0000","last_became_peered":"2024-05-11T12:21:11.534301+0000","last_unstale":"2024-05-11T12:21:42.671237+0000","last_undegraded":"2024-05-11T12:21:42.671237+0000","last_fullsized":"2024-05-11T12:21:42.671237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:24.277988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666399+0000","last_change":"2024-05-11T12:21:08.513794+0000","last_active":"2024-05-11T12:21:42.666399+0000","last_peered":"2024-05-11T12:21:42.666399+0000","last_clean":"2024-05-11T12:21:42.666399+0000","last_became_active":"2024-05-11T12:21:08.513383+0000","last_became_peered":"2024-05-11T12:21:08.513383+0000","last_unstale":"2024-05-11T12:21:42.666399+0000","last_undegraded":"2024-05-11T12:21:42.666399+0000","last_fullsized":"2024-05-11T12:21:42.666399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:50.558886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670055+0000","last_change":"2024-05-11T12:21:11.537147+0000","last_active":"2024-05-11T12:21:42.670055+0000","last_peered":"2024-05-11T12:21:42.670055+0000","last_clean":"2024-05-11T12:21:42.670055+0000","last_became_active":"2024-05-11T12:21:11.536739+0000","last_became_peered":"2024-05-11T12:21:11.536739+0000","last_unstale":"2024-05-11T12:21:42.670055+0000","last_undegraded":"2024-05-11T12:21:42.670055+0000","last_fullsized":"2024-05-11T12:21:42.670055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:00:45.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670989+0000","last_change":"2024-05-11T12:21:08.513494+0000","last_active":"2024-05-11T12:21:42.670989+0000","last_peered":"2024-05-11T12:21:42.670989+0000","last_clean":"2024-05-11T12:21:42.670989+0000","last_became_active":"2024-05-11T12:21:08.513089+0000","last_became_peered":"2024-05-11T12:21:08.513089+0000","last_unstale":"2024-05-11T12:21:42.670989+0000","last_undegraded":"2024-05-11T12:21:42.670989+0000","last_fullsized":"2024-05-11T12:21:42.670989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:48:28.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665885+0000","last_change":"2024-05-11T12:21:11.527827+0000","last_active":"2024-05-11T12:21:42.665885+0000","last_peered":"2024-05-11T12:21:42.665885+0000","last_clean":"2024-05-11T12:21:42.665885+0000","last_became_active":"2024-05-11T12:21:11.527397+0000","last_became_peered":"2024-05-11T12:21:11.527397+0000","last_unstale":"2024-05-11T12:21:42.665885+0000","last_undegraded":"2024-05-11T12:21:42.665885+0000","last_fullsized":"2024-05-11T12:21:42.665885+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:18.710855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662462+0000","last_change":"2024-05-11T12:21:08.513519+0000","last_active":"2024-05-11T12:21:42.662462+0000","last_peered":"2024-05-11T12:21:42.662462+0000","last_clean":"2024-05-11T12:21:42.662462+0000","last_became_active":"2024-05-11T12:21:08.512566+0000","last_became_peered":"2024-05-11T12:21:08.512566+0000","last_unstale":"2024-05-11T12:21:42.662462+0000","last_undegraded":"2024-05-11T12:21:42.662462+0000","last_fullsized":"2024-05-11T12:21:42.662462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:56.104043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671126+0000","last_change":"2024-05-11T12:21:11.535364+0000","last_active":"2024-05-11T12:21:42.671126+0000","last_peered":"2024-05-11T12:21:42.671126+0000","last_clean":"2024-05-11T12:21:42.671126+0000","last_became_active":"2024-05-11T12:21:11.534988+0000","last_became_peered":"2024-05-11T12:21:11.534988+0000","last_unstale":"2024-05-11T12:21:42.671126+0000","last_undegraded":"2024-05-11T12:21:42.671126+0000","last_fullsized":"2024-05-11T12:21:42.671126+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:25:20.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661686+0000","last_change":"2024-05-11T12:21:08.509913+0000","last_active":"2024-05-11T12:21:42.661686+0000","last_peered":"2024-05-11T12:21:42.661686+0000","last_clean":"2024-05-11T12:21:42.661686+0000","last_became_active":"2024-05-11T12:21:08.509498+0000","last_became_peered":"2024-05-11T12:21:08.509498+0000","last_unstale":"2024-05-11T12:21:42.661686+0000","last_undegraded":"2024-05-11T12:21:42.661686+0000","last_fullsized":"2024-05-11T12:21:42.661686+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:43.282700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663416+0000","last_change":"2024-05-11T12:21:11.533108+0000","last_active":"2024-05-11T12:21:42.663416+0000","last_peered":"2024-05-11T12:21:42.663416+0000","last_clean":"2024-05-11T12:21:42.663416+0000","last_became_active":"2024-05-11T12:21:11.532779+0000","last_became_peered":"2024-05-11T12:21:11.532779+0000","last_unstale":"2024-05-11T12:21:42.663416+0000","last_undegraded":"2024-05-11T12:21:42.663416+0000","last_fullsized":"2024-05-11T12:21:42.663416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:55.311786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669903+0000","last_change":"2024-05-11T12:21:08.513355+0000","last_active":"2024-05-11T12:21:42.669903+0000","last_peered":"2024-05-11T12:21:42.669903+0000","last_clean":"2024-05-11T12:21:42.669903+0000","last_became_active":"2024-05-11T12:21:08.512964+0000","last_became_peered":"2024-05-11T12:21:08.512964+0000","last_unstale":"2024-05-11T12:21:42.669903+0000","last_undegraded":"2024-05-11T12:21:42.669903+0000","last_fullsized":"2024-05-11T12:21:42.669903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:12.296865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669837+0000","last_change":"2024-05-11T12:21:11.525958+0000","last_active":"2024-05-11T12:21:42.669837+0000","last_peered":"2024-05-11T12:21:42.669837+0000","last_clean":"2024-05-11T12:21:42.669837+0000","last_became_active":"2024-05-11T12:21:11.525285+0000","last_became_peered":"2024-05-11T12:21:11.525285+0000","last_unstale":"2024-05-11T12:21:42.669837+0000","last_undegraded":"2024-05-11T12:21:42.669837+0000","last_fullsized":"2024-05-11T12:21:42.669837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:24:47.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.673378+0000","last_change":"2024-05-11T12:21:42.673378+0000","last_active":"2024-05-11T12:21:42.673378+0000","last_peered":"2024-05-11T12:21:42.673378+0000","last_clean":"2024-05-11T12:21:42.673378+0000","last_became_active":"2024-05-11T12:21:42.672870+0000","last_became_peered":"2024-05-11T12:21:42.672870+0000","last_unstale":"2024-05-11T12:21:42.673378+0000","last_undegraded":"2024-05-11T12:21:42.673378+0000","last_fullsized":"2024-05-11T12:21:42.673378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:55:41.845162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671927+0000","last_change":"2024-05-11T12:21:11.538946+0000","last_active":"2024-05-11T12:21:42.671927+0000","last_peered":"2024-05-11T12:21:42.671927+0000","last_clean":"2024-05-11T12:21:42.671927+0000","last_became_active":"2024-05-11T12:21:11.535804+0000","last_became_peered":"2024-05-11T12:21:11.535804+0000","last_unstale":"2024-05-11T12:21:42.671927+0000","last_undegraded":"2024-05-11T12:21:42.671927+0000","last_fullsized":"2024-05-11T12:21:42.671927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:37:16.221144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.674203+0000","last_change":"2024-05-11T12:21:42.674203+0000","last_active":"2024-05-11T12:21:42.674203+0000","last_peered":"2024-05-11T12:21:42.674203+0000","last_clean":"2024-05-11T12:21:42.674203+0000","last_became_active":"2024-05-11T12:21:42.673478+0000","last_became_peered":"2024-05-11T12:21:42.673478+0000","last_unstale":"2024-05-11T12:21:42.674203+0000","last_undegraded":"2024-05-11T12:21:42.674203+0000","last_fullsized":"2024-05-11T12:21:42.674203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:20.648528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664568+0000","last_change":"2024-05-11T12:21:11.535346+0000","last_active":"2024-05-11T12:21:42.664568+0000","last_peered":"2024-05-11T12:21:42.664568+0000","last_clean":"2024-05-11T12:21:42.664568+0000","last_became_active":"2024-05-11T12:21:11.530262+0000","last_became_peered":"2024-05-11T12:21:11.530262+0000","last_unstale":"2024-05-11T12:21:42.664568+0000","last_undegraded":"2024-05-11T12:21:42.664568+0000","last_fullsized":"2024-05-11T12:21:42.664568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:13.169797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'3","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670177+0000","last_change":"2024-05-11T12:21:08.531598+0000","last_active":"2024-05-11T12:21:42.670177+0000","last_peered":"2024-05-11T12:21:42.670177+0000","last_clean":"2024-05-11T12:21:42.670177+0000","last_became_active":"2024-05-11T12:21:08.531158+0000","last_became_peered":"2024-05-11T12:21:08.531158+0000","last_unstale":"2024-05-11T12:21:42.670177+0000","last_undegraded":"2024-05-11T12:21:42.670177+0000","last_fullsized":"2024-05-11T12:21:42.670177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:43.528151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671482+0000","last_change":"2024-05-11T12:21:11.539001+0000","last_active":"2024-05-11T12:21:42.671482+0000","last_peered":"2024-05-11T12:21:42.671482+0000","last_clean":"2024-05-11T12:21:42.671482+0000","last_became_active":"2024-05-11T12:21:11.537324+0000","last_became_peered":"2024-05-11T12:21:11.537324+0000","last_unstale":"2024-05-11T12:21:42.671482+0000","last_undegraded":"2024-05-11T12:21:42.671482+0000","last_fullsized":"2024-05-11T12:21:42.671482+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:03:49.393850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":119,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:54.675645+0000","last_change":"2024-05-11T12:21:08.516737+0000","last_active":"2024-05-11T12:21:54.675645+0000","last_peered":"2024-05-11T12:21:54.675645+0000","last_clean":"2024-05-11T12:21:54.675645+0000","last_became_active":"2024-05-11T12:21:08.516316+0000","last_became_peered":"2024-05-11T12:21:08.516316+0000","last_unstale":"2024-05-11T12:21:54.675645+0000","last_undegraded":"2024-05-11T12:21:54.675645+0000","last_fullsized":"2024-05-11T12:21:54.675645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:25.769713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":65,"num_read_kb":92,"num_write":28,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665479+0000","last_change":"2024-05-11T12:21:11.528553+0000","last_active":"2024-05-11T12:21:42.665479+0000","last_peered":"2024-05-11T12:21:42.665479+0000","last_clean":"2024-05-11T12:21:42.665479+0000","last_became_active":"2024-05-11T12:21:11.528175+0000","last_became_peered":"2024-05-11T12:21:11.528175+0000","last_unstale":"2024-05-11T12:21:42.665479+0000","last_undegraded":"2024-05-11T12:21:42.665479+0000","last_fullsized":"2024-05-11T12:21:42.665479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:46:42.689851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'2","reported_seq":17,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.673360+0000","last_change":"2024-05-11T12:21:42.673360+0000","last_active":"2024-05-11T12:21:42.673360+0000","last_peered":"2024-05-11T12:21:42.673360+0000","last_clean":"2024-05-11T12:21:42.673360+0000","last_became_active":"2024-05-11T12:21:42.664842+0000","last_became_peered":"2024-05-11T12:21:42.664842+0000","last_unstale":"2024-05-11T12:21:42.673360+0000","last_undegraded":"2024-05-11T12:21:42.673360+0000","last_fullsized":"2024-05-11T12:21:42.673360+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:40.271658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661675+0000","last_change":"2024-05-11T12:21:11.533157+0000","last_active":"2024-05-11T12:21:42.661675+0000","last_peered":"2024-05-11T12:21:42.661675+0000","last_clean":"2024-05-11T12:21:42.661675+0000","last_became_active":"2024-05-11T12:21:11.532814+0000","last_became_peered":"2024-05-11T12:21:11.532814+0000","last_unstale":"2024-05-11T12:21:42.661675+0000","last_undegraded":"2024-05-11T12:21:42.661675+0000","last_fullsized":"2024-05-11T12:21:42.661675+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:30:56.555384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670629+0000","last_change":"2024-05-11T12:21:08.511907+0000","last_active":"2024-05-11T12:21:42.670629+0000","last_peered":"2024-05-11T12:21:42.670629+0000","last_clean":"2024-05-11T12:21:42.670629+0000","last_became_active":"2024-05-11T12:21:08.511296+0000","last_became_peered":"2024-05-11T12:21:08.511296+0000","last_unstale":"2024-05-11T12:21:42.670629+0000","last_undegraded":"2024-05-11T12:21:42.670629+0000","last_fullsized":"2024-05-11T12:21:42.670629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:43:17.374216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666144+0000","last_change":"2024-05-11T12:21:11.537859+0000","last_active":"2024-05-11T12:21:42.666144+0000","last_peered":"2024-05-11T12:21:42.666144+0000","last_clean":"2024-05-11T12:21:42.666144+0000","last_became_active":"2024-05-11T12:21:11.537448+0000","last_became_peered":"2024-05-11T12:21:11.537448+0000","last_unstale":"2024-05-11T12:21:42.666144+0000","last_undegraded":"2024-05-11T12:21:42.666144+0000","last_fullsized":"2024-05-11T12:21:42.666144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:09:33.866539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670748+0000","last_change":"2024-05-11T12:21:08.510228+0000","last_active":"2024-05-11T12:21:42.670748+0000","last_peered":"2024-05-11T12:21:42.670748+0000","last_clean":"2024-05-11T12:21:42.670748+0000","last_became_active":"2024-05-11T12:21:08.509671+0000","last_became_peered":"2024-05-11T12:21:08.509671+0000","last_unstale":"2024-05-11T12:21:42.670748+0000","last_undegraded":"2024-05-11T12:21:42.670748+0000","last_fullsized":"2024-05-11T12:21:42.670748+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:06.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665875+0000","last_change":"2024-05-11T12:21:11.530935+0000","last_active":"2024-05-11T12:21:42.665875+0000","last_peered":"2024-05-11T12:21:42.665875+0000","last_clean":"2024-05-11T12:21:42.665875+0000","last_became_active":"2024-05-11T12:21:11.530579+0000","last_became_peered":"2024-05-11T12:21:11.530579+0000","last_unstale":"2024-05-11T12:21:42.665875+0000","last_undegraded":"2024-05-11T12:21:42.665875+0000","last_fullsized":"2024-05-11T12:21:42.665875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:38.296021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'8","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.681114+0000","last_change":"2024-05-11T12:21:42.681114+0000","last_active":"2024-05-11T12:21:42.681114+0000","last_peered":"2024-05-11T12:21:42.681114+0000","last_clean":"2024-05-11T12:21:42.681114+0000","last_became_active":"2024-05-11T12:21:42.666878+0000","last_became_peered":"2024-05-11T12:21:42.666878+0000","last_unstale":"2024-05-11T12:21:42.681114+0000","last_undegraded":"2024-05-11T12:21:42.681114+0000","last_fullsized":"2024-05-11T12:21:42.681114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:45.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670655+0000","last_change":"2024-05-11T12:21:11.537879+0000","last_active":"2024-05-11T12:21:42.670655+0000","last_peered":"2024-05-11T12:21:42.670655+0000","last_clean":"2024-05-11T12:21:42.670655+0000","last_became_active":"2024-05-11T12:21:11.537516+0000","last_became_peered":"2024-05-11T12:21:11.537516+0000","last_unstale":"2024-05-11T12:21:42.670655+0000","last_undegraded":"2024-05-11T12:21:42.670655+0000","last_fullsized":"2024-05-11T12:21:42.670655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:05:22.302157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666195+0000","last_change":"2024-05-11T12:21:08.510504+0000","last_active":"2024-05-11T12:21:42.666195+0000","last_peered":"2024-05-11T12:21:42.666195+0000","last_clean":"2024-05-11T12:21:42.666195+0000","last_became_active":"2024-05-11T12:21:08.510074+0000","last_became_peered":"2024-05-11T12:21:08.510074+0000","last_unstale":"2024-05-11T12:21:42.666195+0000","last_undegraded":"2024-05-11T12:21:42.666195+0000","last_fullsized":"2024-05-11T12:21:42.666195+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:31:49.465252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662223+0000","last_change":"2024-05-11T12:21:11.535680+0000","last_active":"2024-05-11T12:21:42.662223+0000","last_peered":"2024-05-11T12:21:42.662223+0000","last_clean":"2024-05-11T12:21:42.662223+0000","last_became_active":"2024-05-11T12:21:11.535292+0000","last_became_peered":"2024-05-11T12:21:11.535292+0000","last_unstale":"2024-05-11T12:21:42.662223+0000","last_undegraded":"2024-05-11T12:21:42.662223+0000","last_fullsized":"2024-05-11T12:21:42.662223+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:57:59.122855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669511+0000","last_change":"2024-05-11T12:21:08.531926+0000","last_active":"2024-05-11T12:21:42.669511+0000","last_peered":"2024-05-11T12:21:42.669511+0000","last_clean":"2024-05-11T12:21:42.669511+0000","last_became_active":"2024-05-11T12:21:08.531548+0000","last_became_peered":"2024-05-11T12:21:08.531548+0000","last_unstale":"2024-05-11T12:21:42.669511+0000","last_undegraded":"2024-05-11T12:21:42.669511+0000","last_fullsized":"2024-05-11T12:21:42.669511+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:25:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662360+0000","last_change":"2024-05-11T12:21:11.528595+0000","last_active":"2024-05-11T12:21:42.662360+0000","last_peered":"2024-05-11T12:21:42.662360+0000","last_clean":"2024-05-11T12:21:42.662360+0000","last_became_active":"2024-05-11T12:21:11.527782+0000","last_became_peered":"2024-05-11T12:21:11.527782+0000","last_unstale":"2024-05-11T12:21:42.662360+0000","last_undegraded":"2024-05-11T12:21:42.662360+0000","last_fullsized":"2024-05-11T12:21:42.662360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:31:00.337497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666277+0000","last_change":"2024-05-11T12:21:08.508093+0000","last_active":"2024-05-11T12:21:42.666277+0000","last_peered":"2024-05-11T12:21:42.666277+0000","last_clean":"2024-05-11T12:21:42.666277+0000","last_became_active":"2024-05-11T12:21:08.507710+0000","last_became_peered":"2024-05-11T12:21:08.507710+0000","last_unstale":"2024-05-11T12:21:42.666277+0000","last_undegraded":"2024-05-11T12:21:42.666277+0000","last_fullsized":"2024-05-11T12:21:42.666277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:24.273932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662325+0000","last_change":"2024-05-11T12:21:11.526026+0000","last_active":"2024-05-11T12:21:42.662325+0000","last_peered":"2024-05-11T12:21:42.662325+0000","last_clean":"2024-05-11T12:21:42.662325+0000","last_became_active":"2024-05-11T12:21:11.525628+0000","last_became_peered":"2024-05-11T12:21:11.525628+0000","last_unstale":"2024-05-11T12:21:42.662325+0000","last_undegraded":"2024-05-11T12:21:42.662325+0000","last_fullsized":"2024-05-11T12:21:42.662325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:12.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670218+0000","last_change":"2024-05-11T12:21:11.535808+0000","last_active":"2024-05-11T12:21:42.670218+0000","last_peered":"2024-05-11T12:21:42.670218+0000","last_clean":"2024-05-11T12:21:42.670218+0000","last_became_active":"2024-05-11T12:21:11.535204+0000","last_became_peered":"2024-05-11T12:21:11.535204+0000","last_unstale":"2024-05-11T12:21:42.670218+0000","last_undegraded":"2024-05-11T12:21:42.670218+0000","last_fullsized":"2024-05-11T12:21:42.670218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:36:01.208543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661581+0000","last_change":"2024-05-11T12:21:08.531838+0000","last_active":"2024-05-11T12:21:42.661581+0000","last_peered":"2024-05-11T12:21:42.661581+0000","last_clean":"2024-05-11T12:21:42.661581+0000","last_became_active":"2024-05-11T12:21:08.531491+0000","last_became_peered":"2024-05-11T12:21:08.531491+0000","last_unstale":"2024-05-11T12:21:42.661581+0000","last_undegraded":"2024-05-11T12:21:42.661581+0000","last_fullsized":"2024-05-11T12:21:42.661581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:33.928985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663841+0000","last_change":"2024-05-11T12:21:11.538563+0000","last_active":"2024-05-11T12:21:42.663841+0000","last_peered":"2024-05-11T12:21:42.663841+0000","last_clean":"2024-05-11T12:21:42.663841+0000","last_became_active":"2024-05-11T12:21:11.538200+0000","last_became_peered":"2024-05-11T12:21:11.538200+0000","last_unstale":"2024-05-11T12:21:42.663841+0000","last_undegraded":"2024-05-11T12:21:42.663841+0000","last_fullsized":"2024-05-11T12:21:42.663841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:18.879528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.672698+0000","last_change":"2024-05-11T12:21:08.509972+0000","last_active":"2024-05-11T12:21:42.672698+0000","last_peered":"2024-05-11T12:21:42.672698+0000","last_clean":"2024-05-11T12:21:42.672698+0000","last_became_active":"2024-05-11T12:21:08.509531+0000","last_became_peered":"2024-05-11T12:21:08.509531+0000","last_unstale":"2024-05-11T12:21:42.672698+0000","last_undegraded":"2024-05-11T12:21:42.672698+0000","last_fullsized":"2024-05-11T12:21:42.672698+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:47:13.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":100,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665982+0000","last_change":"2024-05-11T12:19:42.546860+0000","last_active":"2024-05-11T12:21:42.665982+0000","last_peered":"2024-05-11T12:21:42.665982+0000","last_clean":"2024-05-11T12:21:42.665982+0000","last_became_active":"2024-05-11T12:19:42.541449+0000","last_became_peered":"2024-05-11T12:19:42.541449+0000","last_unstale":"2024-05-11T12:21:42.665982+0000","last_undegraded":"2024-05-11T12:21:42.665982+0000","last_fullsized":"2024-05-11T12:21:42.665982+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_clean_scrub_stamp":"2024-05-11T12:16:12.381187+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:45.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661556+0000","last_change":"2024-05-11T12:21:11.536770+0000","last_active":"2024-05-11T12:21:42.661556+0000","last_peered":"2024-05-11T12:21:42.661556+0000","last_clean":"2024-05-11T12:21:42.661556+0000","last_became_active":"2024-05-11T12:21:11.536421+0000","last_became_peered":"2024-05-11T12:21:11.536421+0000","last_unstale":"2024-05-11T12:21:42.661556+0000","last_undegraded":"2024-05-11T12:21:42.661556+0000","last_fullsized":"2024-05-11T12:21:42.661556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:42.217398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.675283+0000","last_change":"2024-05-11T12:21:42.675283+0000","last_active":"2024-05-11T12:21:42.675283+0000","last_peered":"2024-05-11T12:21:42.675283+0000","last_clean":"2024-05-11T12:21:42.675283+0000","last_became_active":"2024-05-11T12:21:42.674775+0000","last_became_peered":"2024-05-11T12:21:42.674775+0000","last_unstale":"2024-05-11T12:21:42.675283+0000","last_undegraded":"2024-05-11T12:21:42.675283+0000","last_fullsized":"2024-05-11T12:21:42.675283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:13:30.143988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665594+0000","last_change":"2024-05-11T12:21:08.533462+0000","last_active":"2024-05-11T12:21:42.665594+0000","last_peered":"2024-05-11T12:21:42.665594+0000","last_clean":"2024-05-11T12:21:42.665594+0000","last_became_active":"2024-05-11T12:21:08.533081+0000","last_became_peered":"2024-05-11T12:21:08.533081+0000","last_unstale":"2024-05-11T12:21:42.665594+0000","last_undegraded":"2024-05-11T12:21:42.665594+0000","last_fullsized":"2024-05-11T12:21:42.665594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:01:48.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661904+0000","last_change":"2024-05-11T12:21:11.539484+0000","last_active":"2024-05-11T12:21:42.661904+0000","last_peered":"2024-05-11T12:21:42.661904+0000","last_clean":"2024-05-11T12:21:42.661904+0000","last_became_active":"2024-05-11T12:21:11.536810+0000","last_became_peered":"2024-05-11T12:21:11.536810+0000","last_unstale":"2024-05-11T12:21:42.661904+0000","last_undegraded":"2024-05-11T12:21:42.661904+0000","last_fullsized":"2024-05-11T12:21:42.661904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:47:41.461766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:47.602721+0000","last_change":"2024-05-11T12:21:08.510519+0000","last_active":"2024-05-11T12:21:47.602721+0000","last_peered":"2024-05-11T12:21:47.602721+0000","last_clean":"2024-05-11T12:21:47.602721+0000","last_became_active":"2024-05-11T12:21:08.510180+0000","last_became_peered":"2024-05-11T12:21:08.510180+0000","last_unstale":"2024-05-11T12:21:47.602721+0000","last_undegraded":"2024-05-11T12:21:47.602721+0000","last_fullsized":"2024-05-11T12:21:47.602721+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:24.537043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664175+0000","last_change":"2024-05-11T12:21:11.533726+0000","last_active":"2024-05-11T12:21:42.664175+0000","last_peered":"2024-05-11T12:21:42.664175+0000","last_clean":"2024-05-11T12:21:42.664175+0000","last_became_active":"2024-05-11T12:21:11.533393+0000","last_became_peered":"2024-05-11T12:21:11.533393+0000","last_unstale":"2024-05-11T12:21:42.664175+0000","last_undegraded":"2024-05-11T12:21:42.664175+0000","last_fullsized":"2024-05-11T12:21:42.664175+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:03.071666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'2","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670305+0000","last_change":"2024-05-11T12:21:09.036622+0000","last_active":"2024-05-11T12:21:42.670305+0000","last_peered":"2024-05-11T12:21:42.670305+0000","last_clean":"2024-05-11T12:21:42.670305+0000","last_became_active":"2024-05-11T12:21:09.036209+0000","last_became_peered":"2024-05-11T12:21:09.036209+0000","last_unstale":"2024-05-11T12:21:42.670305+0000","last_undegraded":"2024-05-11T12:21:42.670305+0000","last_fullsized":"2024-05-11T12:21:42.670305+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:29:58.947588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665811+0000","last_change":"2024-05-11T12:21:11.531751+0000","last_active":"2024-05-11T12:21:42.665811+0000","last_peered":"2024-05-11T12:21:42.665811+0000","last_clean":"2024-05-11T12:21:42.665811+0000","last_became_active":"2024-05-11T12:21:11.531071+0000","last_became_peered":"2024-05-11T12:21:11.531071+0000","last_unstale":"2024-05-11T12:21:42.665811+0000","last_undegraded":"2024-05-11T12:21:42.665811+0000","last_fullsized":"2024-05-11T12:21:42.665811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:04:48.610370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.671290+0000","last_change":"2024-05-11T12:21:08.533435+0000","last_active":"2024-05-11T12:21:42.671290+0000","last_peered":"2024-05-11T12:21:42.671290+0000","last_clean":"2024-05-11T12:21:42.671290+0000","last_became_active":"2024-05-11T12:21:08.533100+0000","last_became_peered":"2024-05-11T12:21:08.533100+0000","last_unstale":"2024-05-11T12:21:42.671290+0000","last_undegraded":"2024-05-11T12:21:42.671290+0000","last_fullsized":"2024-05-11T12:21:42.671290+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:17.619015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.667054+0000","last_change":"2024-05-11T12:21:11.526140+0000","last_active":"2024-05-11T12:21:42.667054+0000","last_peered":"2024-05-11T12:21:42.667054+0000","last_clean":"2024-05-11T12:21:42.667054+0000","last_became_active":"2024-05-11T12:21:11.525810+0000","last_became_peered":"2024-05-11T12:21:11.525810+0000","last_unstale":"2024-05-11T12:21:42.667054+0000","last_undegraded":"2024-05-11T12:21:42.667054+0000","last_fullsized":"2024-05-11T12:21:42.667054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:14:04.160379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664274+0000","last_change":"2024-05-11T12:21:08.508031+0000","last_active":"2024-05-11T12:21:42.664274+0000","last_peered":"2024-05-11T12:21:42.664274+0000","last_clean":"2024-05-11T12:21:42.664274+0000","last_became_active":"2024-05-11T12:21:08.507703+0000","last_became_peered":"2024-05-11T12:21:08.507703+0000","last_unstale":"2024-05-11T12:21:42.664274+0000","last_undegraded":"2024-05-11T12:21:42.664274+0000","last_fullsized":"2024-05-11T12:21:42.664274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:05.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661987+0000","last_change":"2024-05-11T12:21:11.528293+0000","last_active":"2024-05-11T12:21:42.661987+0000","last_peered":"2024-05-11T12:21:42.661987+0000","last_clean":"2024-05-11T12:21:42.661987+0000","last_became_active":"2024-05-11T12:21:11.527862+0000","last_became_peered":"2024-05-11T12:21:11.527862+0000","last_unstale":"2024-05-11T12:21:42.661987+0000","last_undegraded":"2024-05-11T12:21:42.661987+0000","last_fullsized":"2024-05-11T12:21:42.661987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:07.335829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665126+0000","last_change":"2024-05-11T12:21:08.510138+0000","last_active":"2024-05-11T12:21:42.665126+0000","last_peered":"2024-05-11T12:21:42.665126+0000","last_clean":"2024-05-11T12:21:42.665126+0000","last_became_active":"2024-05-11T12:21:08.509566+0000","last_became_peered":"2024-05-11T12:21:08.509566+0000","last_unstale":"2024-05-11T12:21:42.665126+0000","last_undegraded":"2024-05-11T12:21:42.665126+0000","last_fullsized":"2024-05-11T12:21:42.665126+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:24:58.174305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663889+0000","last_change":"2024-05-11T12:21:11.528730+0000","last_active":"2024-05-11T12:21:42.663889+0000","last_peered":"2024-05-11T12:21:42.663889+0000","last_clean":"2024-05-11T12:21:42.663889+0000","last_became_active":"2024-05-11T12:21:11.528355+0000","last_became_peered":"2024-05-11T12:21:11.528355+0000","last_unstale":"2024-05-11T12:21:42.663889+0000","last_undegraded":"2024-05-11T12:21:42.663889+0000","last_fullsized":"2024-05-11T12:21:42.663889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:49:07.559329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"79'2","reported_seq":39,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.683706+0000","last_change":"2024-05-11T12:21:42.683706+0000","last_active":"2024-05-11T12:21:42.683706+0000","last_peered":"2024-05-11T12:21:42.683706+0000","last_clean":"2024-05-11T12:21:42.683706+0000","last_became_active":"2024-05-11T12:21:42.675791+0000","last_became_peered":"2024-05-11T12:21:42.675791+0000","last_unstale":"2024-05-11T12:21:42.683706+0000","last_undegraded":"2024-05-11T12:21:42.683706+0000","last_fullsized":"2024-05-11T12:21:42.683706+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:13:26.737525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4],"acting":[8,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666251+0000","last_change":"2024-05-11T12:21:11.544153+0000","last_active":"2024-05-11T12:21:42.666251+0000","last_peered":"2024-05-11T12:21:42.666251+0000","last_clean":"2024-05-11T12:21:42.666251+0000","last_became_active":"2024-05-11T12:21:11.543772+0000","last_became_peered":"2024-05-11T12:21:11.543772+0000","last_unstale":"2024-05-11T12:21:42.666251+0000","last_undegraded":"2024-05-11T12:21:42.666251+0000","last_fullsized":"2024-05-11T12:21:42.666251+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:54:30.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665919+0000","last_change":"2024-05-11T12:21:09.036409+0000","last_active":"2024-05-11T12:21:42.665919+0000","last_peered":"2024-05-11T12:21:42.665919+0000","last_clean":"2024-05-11T12:21:42.665919+0000","last_became_active":"2024-05-11T12:21:09.036009+0000","last_became_peered":"2024-05-11T12:21:09.036009+0000","last_unstale":"2024-05-11T12:21:42.665919+0000","last_undegraded":"2024-05-11T12:21:42.665919+0000","last_fullsized":"2024-05-11T12:21:42.665919+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:55:28.736718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666612+0000","last_change":"2024-05-11T12:21:11.542685+0000","last_active":"2024-05-11T12:21:42.666612+0000","last_peered":"2024-05-11T12:21:42.666612+0000","last_clean":"2024-05-11T12:21:42.666612+0000","last_became_active":"2024-05-11T12:21:11.542130+0000","last_became_peered":"2024-05-11T12:21:11.542130+0000","last_unstale":"2024-05-11T12:21:42.666612+0000","last_undegraded":"2024-05-11T12:21:42.666612+0000","last_fullsized":"2024-05-11T12:21:42.666612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:22.578809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.669920+0000","last_change":"2024-05-11T12:21:08.510869+0000","last_active":"2024-05-11T12:21:42.669920+0000","last_peered":"2024-05-11T12:21:42.669920+0000","last_clean":"2024-05-11T12:21:42.669920+0000","last_became_active":"2024-05-11T12:21:08.510510+0000","last_became_peered":"2024-05-11T12:21:08.510510+0000","last_unstale":"2024-05-11T12:21:42.669920+0000","last_undegraded":"2024-05-11T12:21:42.669920+0000","last_fullsized":"2024-05-11T12:21:42.669920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:50.002313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666183+0000","last_change":"2024-05-11T12:21:11.538622+0000","last_active":"2024-05-11T12:21:42.666183+0000","last_peered":"2024-05-11T12:21:42.666183+0000","last_clean":"2024-05-11T12:21:42.666183+0000","last_became_active":"2024-05-11T12:21:11.538229+0000","last_became_peered":"2024-05-11T12:21:11.538229+0000","last_unstale":"2024-05-11T12:21:42.666183+0000","last_undegraded":"2024-05-11T12:21:42.666183+0000","last_fullsized":"2024-05-11T12:21:42.666183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:01:15.912428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":38,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.677421+0000","last_change":"2024-05-11T12:21:42.677421+0000","last_active":"2024-05-11T12:21:42.677421+0000","last_peered":"2024-05-11T12:21:42.677421+0000","last_clean":"2024-05-11T12:21:42.677421+0000","last_became_active":"2024-05-11T12:21:42.672695+0000","last_became_peered":"2024-05-11T12:21:42.672695+0000","last_unstale":"2024-05-11T12:21:42.677421+0000","last_undegraded":"2024-05-11T12:21:42.677421+0000","last_fullsized":"2024-05-11T12:21:42.677421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:30:27.606816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666671+0000","last_change":"2024-05-11T12:21:11.536149+0000","last_active":"2024-05-11T12:21:42.666671+0000","last_peered":"2024-05-11T12:21:42.666671+0000","last_clean":"2024-05-11T12:21:42.666671+0000","last_became_active":"2024-05-11T12:21:11.535763+0000","last_became_peered":"2024-05-11T12:21:11.535763+0000","last_unstale":"2024-05-11T12:21:42.666671+0000","last_undegraded":"2024-05-11T12:21:42.666671+0000","last_fullsized":"2024-05-11T12:21:42.666671+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:46:15.246610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662148+0000","last_change":"2024-05-11T12:21:11.534318+0000","last_active":"2024-05-11T12:21:42.662148+0000","last_peered":"2024-05-11T12:21:42.662148+0000","last_clean":"2024-05-11T12:21:42.662148+0000","last_became_active":"2024-05-11T12:21:11.533949+0000","last_became_peered":"2024-05-11T12:21:11.533949+0000","last_unstale":"2024-05-11T12:21:42.662148+0000","last_undegraded":"2024-05-11T12:21:42.662148+0000","last_fullsized":"2024-05-11T12:21:42.662148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:41:42.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665121+0000","last_change":"2024-05-11T12:21:11.529276+0000","last_active":"2024-05-11T12:21:42.665121+0000","last_peered":"2024-05-11T12:21:42.665121+0000","last_clean":"2024-05-11T12:21:42.665121+0000","last_became_active":"2024-05-11T12:21:11.528857+0000","last_became_peered":"2024-05-11T12:21:11.528857+0000","last_unstale":"2024-05-11T12:21:42.665121+0000","last_undegraded":"2024-05-11T12:21:42.665121+0000","last_fullsized":"2024-05-11T12:21:42.665121+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:54.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664991+0000","last_change":"2024-05-11T12:21:11.532342+0000","last_active":"2024-05-11T12:21:42.664991+0000","last_peered":"2024-05-11T12:21:42.664991+0000","last_clean":"2024-05-11T12:21:42.664991+0000","last_became_active":"2024-05-11T12:21:11.531949+0000","last_became_peered":"2024-05-11T12:21:11.531949+0000","last_unstale":"2024-05-11T12:21:42.664991+0000","last_undegraded":"2024-05-11T12:21:42.664991+0000","last_fullsized":"2024-05-11T12:21:42.664991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:23:00.010874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.665522+0000","last_change":"2024-05-11T12:21:11.531547+0000","last_active":"2024-05-11T12:21:42.665522+0000","last_peered":"2024-05-11T12:21:42.665522+0000","last_clean":"2024-05-11T12:21:42.665522+0000","last_became_active":"2024-05-11T12:21:11.529702+0000","last_became_peered":"2024-05-11T12:21:11.529702+0000","last_unstale":"2024-05-11T12:21:42.665522+0000","last_undegraded":"2024-05-11T12:21:42.665522+0000","last_fullsized":"2024-05-11T12:21:42.665522+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:13.857026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661964+0000","last_change":"2024-05-11T12:21:11.531261+0000","last_active":"2024-05-11T12:21:42.661964+0000","last_peered":"2024-05-11T12:21:42.661964+0000","last_clean":"2024-05-11T12:21:42.661964+0000","last_became_active":"2024-05-11T12:21:11.530561+0000","last_became_peered":"2024-05-11T12:21:11.530561+0000","last_unstale":"2024-05-11T12:21:42.661964+0000","last_undegraded":"2024-05-11T12:21:42.661964+0000","last_fullsized":"2024-05-11T12:21:42.661964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:52:07.654417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662182+0000","last_change":"2024-05-11T12:21:11.533255+0000","last_active":"2024-05-11T12:21:42.662182+0000","last_peered":"2024-05-11T12:21:42.662182+0000","last_clean":"2024-05-11T12:21:42.662182+0000","last_became_active":"2024-05-11T12:21:11.532909+0000","last_became_peered":"2024-05-11T12:21:11.532909+0000","last_unstale":"2024-05-11T12:21:42.662182+0000","last_undegraded":"2024-05-11T12:21:42.662182+0000","last_fullsized":"2024-05-11T12:21:42.662182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:07:39.501482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.666539+0000","last_change":"2024-05-11T12:21:11.528856+0000","last_active":"2024-05-11T12:21:42.666539+0000","last_peered":"2024-05-11T12:21:42.666539+0000","last_clean":"2024-05-11T12:21:42.666539+0000","last_became_active":"2024-05-11T12:21:11.528230+0000","last_became_peered":"2024-05-11T12:21:11.528230+0000","last_unstale":"2024-05-11T12:21:42.666539+0000","last_undegraded":"2024-05-11T12:21:42.666539+0000","last_fullsized":"2024-05-11T12:21:42.666539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:40.429045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.664005+0000","last_change":"2024-05-11T12:21:11.534606+0000","last_active":"2024-05-11T12:21:42.664005+0000","last_peered":"2024-05-11T12:21:42.664005+0000","last_clean":"2024-05-11T12:21:42.664005+0000","last_became_active":"2024-05-11T12:21:11.534193+0000","last_became_peered":"2024-05-11T12:21:11.534193+0000","last_unstale":"2024-05-11T12:21:42.664005+0000","last_undegraded":"2024-05-11T12:21:42.664005+0000","last_fullsized":"2024-05-11T12:21:42.664005+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:00:12.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670022+0000","last_change":"2024-05-11T12:21:11.525959+0000","last_active":"2024-05-11T12:21:42.670022+0000","last_peered":"2024-05-11T12:21:42.670022+0000","last_clean":"2024-05-11T12:21:42.670022+0000","last_became_active":"2024-05-11T12:21:11.525129+0000","last_became_peered":"2024-05-11T12:21:11.525129+0000","last_unstale":"2024-05-11T12:21:42.670022+0000","last_undegraded":"2024-05-11T12:21:42.670022+0000","last_fullsized":"2024-05-11T12:21:42.670022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:28.828336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.663994+0000","last_change":"2024-05-11T12:21:11.538591+0000","last_active":"2024-05-11T12:21:42.663994+0000","last_peered":"2024-05-11T12:21:42.663994+0000","last_clean":"2024-05-11T12:21:42.663994+0000","last_became_active":"2024-05-11T12:21:11.537855+0000","last_became_peered":"2024-05-11T12:21:11.537855+0000","last_unstale":"2024-05-11T12:21:42.663994+0000","last_undegraded":"2024-05-11T12:21:42.663994+0000","last_fullsized":"2024-05-11T12:21:42.663994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:52.625763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.670959+0000","last_change":"2024-05-11T12:21:11.541185+0000","last_active":"2024-05-11T12:21:42.670959+0000","last_peered":"2024-05-11T12:21:42.670959+0000","last_clean":"2024-05-11T12:21:42.670959+0000","last_became_active":"2024-05-11T12:21:11.540799+0000","last_became_peered":"2024-05-11T12:21:11.540799+0000","last_unstale":"2024-05-11T12:21:42.670959+0000","last_undegraded":"2024-05-11T12:21:42.670959+0000","last_fullsized":"2024-05-11T12:21:42.670959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:24.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.661882+0000","last_change":"2024-05-11T12:21:11.527811+0000","last_active":"2024-05-11T12:21:42.661882+0000","last_peered":"2024-05-11T12:21:42.661882+0000","last_clean":"2024-05-11T12:21:42.661882+0000","last_became_active":"2024-05-11T12:21:11.526603+0000","last_became_peered":"2024-05-11T12:21:11.526603+0000","last_unstale":"2024-05-11T12:21:42.661882+0000","last_undegraded":"2024-05-11T12:21:42.661882+0000","last_fullsized":"2024-05-11T12:21:42.661882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:43.833506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-05-11T12:21:42.662052+0000","last_change":"2024-05-11T12:21:11.530824+0000","last_active":"2024-05-11T12:21:42.662052+0000","last_peered":"2024-05-11T12:21:42.662052+0000","last_clean":"2024-05-11T12:21:42.662052+0000","last_became_active":"2024-05-11T12:21:11.530486+0000","last_became_peered":"2024-05-11T12:21:11.530486+0000","last_unstale":"2024-05-11T12:21:42.662052+0000","last_undegraded":"2024-05-11T12:21:42.662052+0000","last_fullsized":"2024-05-11T12:21:42.662052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:47.054822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247249,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62548,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":71,"num_read_kb":98,"num_write":85,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":179130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678040,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":427807,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874269,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":1003520,"data_stored":1080202,"data_compressed":41556,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037795,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36188,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708964,"statfs":{"total":95995035648,"available":95957979136,"internally_reserved":0,"allocated":872448,"data_stored":428355,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234023,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":876544,"data_stored":428377,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":925696,"data_stored":1018840,"data_compressed":41556,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593779,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36192,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708960,"statfs":{"total":95995035648,"available":95957975040,"internally_reserved":0,"allocated":872448,"data_stored":427890,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36248,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708904,"statfs":{"total":95995035648,"available":95957917696,"internally_reserved":0,"allocated":933888,"data_stored":485266,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":872448,"data_stored":427863,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":485266,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":884736,"data_stored":428111,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953533,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":427795,"data_compressed":5771,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790007,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":1018440,"data_compressed":41556,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:21:56.739 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-11T12:21:56.740 INFO:tasks.cephadm:Setting up client nodes... 2024-05-11T12:21:56.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-11T12:21:56.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:56 smithi050 ceph-mon[34072]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 4 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:56.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:56 smithi050 ceph-mon[34072]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:21:56.996 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:56 smithi047 ceph-mon[30375]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 4 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:56 smithi047 ceph-mon[30375]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:21:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:56 smithi064 ceph-mon[34537]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 4 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:56 smithi064 ceph-mon[34537]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:21:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 -- 172.21.15.47:0/1024804999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0104060 msgr2=0x7fc3b0106450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/1024804999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0104060 0x7fc3b0106450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000b0a0 tx=0x7fc3a000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 -- 172.21.15.47:0/1024804999 shutdown_connections 2024-05-11T12:21:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/1024804999 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b01092c0 0x7fc3b010b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/1024804999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0104060 0x7fc3b0106450 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/1024804999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3b0106990 0x7fc3b0108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 -- 172.21.15.47:0/1024804999 >> 172.21.15.47:0/1024804999 conn(0x7fc3b00fda70 msgr2=0x7fc3b00ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 -- 172.21.15.47:0/1024804999 shutdown_connections 2024-05-11T12:21:58.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.609+0000 7fc3b5e05700 1 -- 172.21.15.47:0/1024804999 wait complete. 2024-05-11T12:21:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 Processor -- start 2024-05-11T12:21:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 -- start start 2024-05-11T12:21:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3b0104060 0x7fc3b01039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0106990 0x7fc3b0102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b01092c0 0x7fc3b0102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:58.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b010f090 con 0x7fc3b0106990 2024-05-11T12:21:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b010ef10 con 0x7fc3b0104060 2024-05-11T12:21:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3b5e05700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b010f210 con 0x7fc3b01092c0 2024-05-11T12:21:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0106990 0x7fc3b0102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0106990 0x7fc3b0102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47414/0 (socket says 172.21.15.47:47414) 2024-05-11T12:21:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.610+0000 7fc3aeffd700 1 -- 172.21.15.47:0/4032976854 learned_addr learned my addr 172.21.15.47:0/4032976854 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:21:58.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3af7fe700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3b0104060 0x7fc3b01039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3affff700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b01092c0 0x7fc3b0102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3aeffd700 1 -- 172.21.15.47:0/4032976854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b01092c0 msgr2=0x7fc3b0102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3aeffd700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b01092c0 0x7fc3b0102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3aeffd700 1 -- 172.21.15.47:0/4032976854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3b0104060 msgr2=0x7fc3b01039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:58.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3aeffd700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3b0104060 0x7fc3b01039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3aeffd700 1 -- 172.21.15.47:0/4032976854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b0102e00 con 0x7fc3b0106990 2024-05-11T12:21:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3affff700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b01092c0 0x7fc3b0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:21:58.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3af7fe700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3b0104060 0x7fc3b01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:21:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3aeffd700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0106990 0x7fc3b0102010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c9b0 tx=0x7fc39800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3acff9700 1 -- 172.21.15.47:0/4032976854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398007b00 con 0x7fc3b0106990 2024-05-11T12:21:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3acff9700 1 -- 172.21.15.47:0/4032976854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc398007ca0 con 0x7fc3b0106990 2024-05-11T12:21:58.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b01a36f0 con 0x7fc3b0106990 2024-05-11T12:21:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3acff9700 1 -- 172.21.15.47:0/4032976854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398018730 con 0x7fc3b0106990 2024-05-11T12:21:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.611+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b01a3c00 con 0x7fc3b0106990 2024-05-11T12:21:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.612+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b0051a90 con 0x7fc3b0106990 2024-05-11T12:21:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.613+0000 7fc3acff9700 1 -- 172.21.15.47:0/4032976854 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc39800f450 con 0x7fc3b0106990 2024-05-11T12:21:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.614+0000 7fc3acff9700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc39c0b29b0 0x7fc39c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:21:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.614+0000 7fc3af7fe700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc39c0b29b0 0x7fc39c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:21:58.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.614+0000 7fc3acff9700 1 -- 172.21.15.47:0/4032976854 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc3980e2750 con 0x7fc3b0106990 2024-05-11T12:21:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.615+0000 7fc3af7fe700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc39c0b29b0 0x7fc39c0b4e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000b070 tx=0x7fc3a000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:21:58.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.618+0000 7fc3acff9700 1 -- 172.21.15.47:0/4032976854 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc398014030 con 0x7fc3b0106990 2024-05-11T12:21:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.933+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc3b002e570 con 0x7fc3b0106990 2024-05-11T12:21:58.934 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:58 smithi047 ceph-mon[30375]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.935+0000 7fc3acff9700 1 -- 172.21.15.47:0/4032976854 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc3980aa380 con 0x7fc3b0106990 2024-05-11T12:21:58.936 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-05-11T12:21:58.936 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQDmYj9misW+NxAAz1sHPgD1HivVWa16cL2nYg== 2024-05-11T12:21:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc39c0b29b0 msgr2=0x7fc39c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc39c0b29b0 0x7fc39c0b4e70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000b070 tx=0x7fc3a000b7b0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0106990 msgr2=0x7fc3b0102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:21:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0106990 0x7fc3b0102010 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c9b0 tx=0x7fc39800cd70 comp rx=0 tx=0).stop 2024-05-11T12:21:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 shutdown_connections 2024-05-11T12:21:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc3b01092c0 0x7fc3b0102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc39c0b29b0 0x7fc39c0b4e70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.938+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc3b0104060 0x7fc3b01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.939+0000 7fc3b5e05700 1 --2- 172.21.15.47:0/4032976854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc3b0106990 0x7fc3b0102010 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:21:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.939+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 >> 172.21.15.47:0/4032976854 conn(0x7fc3b00fda70 msgr2=0x7fc3b01049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:21:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.939+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 shutdown_connections 2024-05-11T12:21:58.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:21:58.939+0000 7fc3b5e05700 1 -- 172.21.15.47:0/4032976854 wait complete. 2024-05-11T12:21:58.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:58 smithi050 ceph-mon[34072]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:58 smithi064 ceph-mon[34537]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-05-11T12:21:59.587 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:21:59.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-11T12:21:59.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-11T12:21:59.628 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-11T12:21:59.629 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-11T12:21:59.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph mgr dump --format=json 2024-05-11T12:21:59.921 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:21:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:59 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/4032976854' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T12:21:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:21:59 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/4032976854' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-11T12:22:00.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:59 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/4032976854' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T12:22:00.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:21:59 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/4032976854' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-11T12:22:00.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:59 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/4032976854' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-11T12:22:00.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:21:59 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/4032976854' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-11T12:22:01.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:00 smithi047 ceph-mon[30375]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-11T12:22:01.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:00 smithi050 ceph-mon[34072]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-11T12:22:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:00 smithi064 ceph-mon[34537]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-05-11T12:22:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 -- 172.21.15.47:0/608085629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d40760a0 msgr2=0x7f21d410fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/608085629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d40760a0 0x7f21d410fa10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009990 tx=0x7f21c4009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 -- 172.21.15.47:0/608085629 shutdown_connections 2024-05-11T12:22:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/608085629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d410ff50 0x7f21d4112340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/608085629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d4075780 0x7f21d4075b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/608085629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d40760a0 0x7f21d410fa10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 -- 172.21.15.47:0/608085629 >> 172.21.15.47:0/608085629 conn(0x7f21d407a6b0 msgr2=0x7f21d407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 -- 172.21.15.47:0/608085629 shutdown_connections 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.567+0000 7f21dc2d7700 1 -- 172.21.15.47:0/608085629 wait complete. 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 Processor -- start 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 -- start start 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d4075780 0x7f21d419be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d40760a0 0x7f21d419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d410ff50 0x7f21d41a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4115980 con 0x7f21d4075780 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4115800 con 0x7f21d40760a0 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21dc2d7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f21d4115b00 con 0x7f21d410ff50 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da073700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d4075780 0x7f21d419be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da073700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d4075780 0x7f21d419be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47418/0 (socket says 172.21.15.47:47418) 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da073700 1 -- 172.21.15.47:0/3961436963 learned_addr learned my addr 172.21.15.47:0/3961436963 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da874700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d410ff50 0x7f21d41a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21d9872700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d40760a0 0x7f21d419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da073700 1 -- 172.21.15.47:0/3961436963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d410ff50 msgr2=0x7f21d41a0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da073700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d410ff50 0x7f21d41a0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da073700 1 -- 172.21.15.47:0/3961436963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d40760a0 msgr2=0x7f21d419c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.568+0000 7f21da073700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d40760a0 0x7f21d419c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21da073700 1 -- 172.21.15.47:0/3961436963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d41a0e50 con 0x7f21d4075780 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21d9872700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d40760a0 0x7f21d419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21da073700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d4075780 0x7f21d419be90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c920 tx=0x7f21d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21cb7fe700 1 -- 172.21.15.47:0/3961436963 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0007b00 con 0x7f21d4075780 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21cb7fe700 1 -- 172.21.15.47:0/3961436963 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d0004ce0 con 0x7f21d4075780 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d41a1140 con 0x7f21d4075780 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21cb7fe700 1 -- 172.21.15.47:0/3961436963 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0005550 con 0x7f21d4075780 2024-05-11T12:22:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.569+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d41a89c0 con 0x7f21d4075780 2024-05-11T12:22:01.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.570+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d406a240 con 0x7f21d4075780 2024-05-11T12:22:01.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.572+0000 7f21cb7fe700 1 -- 172.21.15.47:0/3961436963 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f21d0007ca0 con 0x7f21d4075780 2024-05-11T12:22:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.573+0000 7f21cb7fe700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f21c00b2990 0x7f21c00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.573+0000 7f21d9872700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f21c00b2990 0x7f21c00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.573+0000 7f21cb7fe700 1 -- 172.21.15.47:0/3961436963 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f21d00e2310 con 0x7f21d4075780 2024-05-11T12:22:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.574+0000 7f21d9872700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f21c00b2990 0x7f21c00b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21c400b5c0 tx=0x7f21c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:01.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.577+0000 7f21cb7fe700 1 -- 172.21.15.47:0/3961436963 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f21d0014030 con 0x7f21d4075780 2024-05-11T12:22:01.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.852+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f21d402e570 con 0x7f21d4075780 2024-05-11T12:22:01.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.857+0000 7f21cb7fe700 1 -- 172.21.15.47:0/3961436963 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288641 (secure 0 0 0) 0x7f21d00a9ed0 con 0x7f21d4075780 2024-05-11T12:22:01.862 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:22:01.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f21c00b2990 msgr2=0x7f21c00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:01.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f21c00b2990 0x7f21c00b4e50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21c400b5c0 tx=0x7f21c4005fb0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d4075780 msgr2=0x7f21d419be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d4075780 0x7f21d419be90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c920 tx=0x7f21d000cc30 comp rx=0 tx=0).stop 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 shutdown_connections 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f21d410ff50 0x7f21d41a0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f21d4075780 0x7f21d419be90 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f21c00b2990 0x7f21c00b4e50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 --2- 172.21.15.47:0/3961436963 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f21d40760a0 0x7f21d419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 >> 172.21.15.47:0/3961436963 conn(0x7f21d407a6b0 msgr2=0x7f21d41007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 shutdown_connections 2024-05-11T12:22:01.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:01.861+0000 7f21dc2d7700 1 -- 172.21.15.47:0/3961436963 wait complete. 2024-05-11T12:22:02.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:01 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3961436963' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-11T12:22:02.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:01 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3961436963' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-11T12:22:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:01 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3961436963' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-11T12:22:02.556 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":811043456},{"type":"v1","addr":"172.21.15.47:6801","nonce":811043456}]},"active_addr":"172.21.15.47:6801/811043456","active_change":"2024-05-11T12:13:41.429919+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24122,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.47:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":91951901}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":4119076698}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3680207492}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":3079808889}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.47:0","nonce":969559970}]}]} 2024-05-11T12:22:02.563 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-11T12:22:02.563 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-11T12:22:02.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:22:02.818 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:03.194 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:02 smithi047 ceph-mon[30375]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-05-11T12:22:03.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:02 smithi050 ceph-mon[34072]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-05-11T12:22:03.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:02 smithi064 ceph-mon[34537]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-05-11T12:22:04.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.487+0000 7f2691ba4700 1 -- 172.21.15.47:0/2932180695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c1041b0 msgr2=0x7f268c1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.487+0000 7f2691ba4700 1 --2- 172.21.15.47:0/2932180695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c1041b0 0x7f268c1081d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f26740099c0 tx=0x7f2674009cd0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.487+0000 7f2691ba4700 1 -- 172.21.15.47:0/2932180695 shutdown_connections 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.487+0000 7f2691ba4700 1 --2- 172.21.15.47:0/2932180695 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f268c108900 0x7f268c110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.487+0000 7f2691ba4700 1 --2- 172.21.15.47:0/2932180695 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c103800 0x7f268c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.487+0000 7f2691ba4700 1 --2- 172.21.15.47:0/2932180695 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c1041b0 0x7f268c1081d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.488+0000 7f2691ba4700 1 -- 172.21.15.47:0/2932180695 >> 172.21.15.47:0/2932180695 conn(0x7f268c07a6b0 msgr2=0x7f268c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.488+0000 7f2691ba4700 1 -- 172.21.15.47:0/2932180695 shutdown_connections 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.488+0000 7f2691ba4700 1 -- 172.21.15.47:0/2932180695 wait complete. 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.488+0000 7f2691ba4700 1 Processor -- start 2024-05-11T12:22:04.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.488+0000 7f2691ba4700 1 -- start start 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f2691ba4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c103800 0x7f268c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f2691ba4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f268c1041b0 0x7f268c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f2691ba4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c108900 0x7f268c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f2691ba4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f268c113730 con 0x7f268c103800 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f2691ba4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1135b0 con 0x7f268c1041b0 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f2691ba4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f268c1138b0 con 0x7f268c108900 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c103800 0x7f268c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c103800 0x7f268c19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47430/0 (socket says 172.21.15.47:47430) 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 -- 172.21.15.47:0/479915853 learned_addr learned my addr 172.21.15.47:0/479915853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268bfff700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c108900 0x7f268c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 -- 172.21.15.47:0/479915853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c108900 msgr2=0x7f268c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c108900 0x7f268c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 -- 172.21.15.47:0/479915853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f268c1041b0 msgr2=0x7f268c19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f268c1041b0 0x7f268c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 -- 172.21.15.47:0/479915853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c19a150 con 0x7f268c103800 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268bfff700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c108900 0x7f268c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f268b7fe700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c103800 0x7f268c19f7e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f267c00c9b0 tx=0x7f267c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.489+0000 7f26897fa700 1 -- 172.21.15.47:0/479915853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c007b00 con 0x7f268c103800 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.490+0000 7f26897fa700 1 -- 172.21.15.47:0/479915853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f267c007ca0 con 0x7f268c103800 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.490+0000 7f26897fa700 1 -- 172.21.15.47:0/479915853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267c018730 con 0x7f268c103800 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.490+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268c19a440 con 0x7f268c103800 2024-05-11T12:22:04.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.490+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268c1a66b0 con 0x7f268c103800 2024-05-11T12:22:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.491+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268c06a240 con 0x7f268c103800 2024-05-11T12:22:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.492+0000 7f26897fa700 1 -- 172.21.15.47:0/479915853 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f267c00f450 con 0x7f268c103800 2024-05-11T12:22:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.493+0000 7f26897fa700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26780b2990 0x7f26780b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.493+0000 7f2683fff700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26780b2990 0x7f26780b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:04.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.493+0000 7f26897fa700 1 -- 172.21.15.47:0/479915853 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f267c0e26d0 con 0x7f268c103800 2024-05-11T12:22:04.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.494+0000 7f2683fff700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26780b2990 0x7f26780b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2674003800 tx=0x7f2674000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:04.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.496+0000 7f26897fa700 1 -- 172.21.15.47:0/479915853 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f267c014030 con 0x7f268c103800 2024-05-11T12:22:04.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.713+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f268c02e570 con 0x7f268c103800 2024-05-11T12:22:04.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.714+0000 7f26897fa700 1 -- 172.21.15.47:0/479915853 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22638 (secure 0 0 0) 0x7f267c0aa380 con 0x7f268c103800 2024-05-11T12:22:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:22:04.715 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":81,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:21:42.658586+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26780b2990 msgr2=0x7f26780b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26780b2990 0x7f26780b4e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2674003800 tx=0x7f2674000bc0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c103800 msgr2=0x7f268c19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c103800 0x7f268c19f7e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f267c00c9b0 tx=0x7f267c00cd70 comp rx=0 tx=0).stop 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 shutdown_connections 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f268c108900 0x7f268c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f268c103800 0x7f268c19f7e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26780b2990 0x7f26780b4e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 --2- 172.21.15.47:0/479915853 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f268c1041b0 0x7f268c19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:04.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 >> 172.21.15.47:0/479915853 conn(0x7f268c07a6b0 msgr2=0x7f268c1017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:04.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 shutdown_connections 2024-05-11T12:22:04.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:04.717+0000 7f2691ba4700 1 -- 172.21.15.47:0/479915853 wait complete. 2024-05-11T12:22:05.173 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:04 smithi047 ceph-mon[30375]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-05-11T12:22:05.174 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:04 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/479915853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:22:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:04 smithi064 ceph-mon[34537]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-05-11T12:22:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:04 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/479915853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:22:05.408 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-11T12:22:05.408 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:22:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:04 smithi050 ceph-mon[34072]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-05-11T12:22:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:04 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/479915853' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:22:05.664 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:07.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:06 smithi047 ceph-mon[30375]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-05-11T12:22:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 -- 172.21.15.47:0/4127212799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c1084c0 msgr2=0x7f587c1088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 --2- 172.21.15.47:0/4127212799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c1084c0 0x7f587c1088a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5878009990 tx=0x7f5878009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 -- 172.21.15.47:0/4127212799 shutdown_connections 2024-05-11T12:22:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 --2- 172.21.15.47:0/4127212799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f587c102e40 0x7f587c10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 --2- 172.21.15.47:0/4127212799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c1084c0 0x7f587c1088a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 --2- 172.21.15.47:0/4127212799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587c102480 0x7f587c102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 -- 172.21.15.47:0/4127212799 >> 172.21.15.47:0/4127212799 conn(0x7f587c079400 msgr2=0x7f587c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 -- 172.21.15.47:0/4127212799 shutdown_connections 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.344+0000 7f58836e3700 1 -- 172.21.15.47:0/4127212799 wait complete. 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 Processor -- start 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 -- start start 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f587c102480 0x7f587c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c102e40 0x7f587c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587c1084c0 0x7f587c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f587c112650 con 0x7f587c102e40 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f587c1124d0 con 0x7f587c102480 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f58836e3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f587c1127d0 con 0x7f587c1084c0 2024-05-11T12:22:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c102e40 0x7f587c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c102e40 0x7f587c19fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34638/0 (socket says 172.21.15.47:34638) 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 -- 172.21.15.47:0/1430599404 learned_addr learned my addr 172.21.15.47:0/1430599404 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f588147f700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f587c102480 0x7f587c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5881c80700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587c1084c0 0x7f587c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 -- 172.21.15.47:0/1430599404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587c1084c0 msgr2=0x7f587c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587c1084c0 0x7f587c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 -- 172.21.15.47:0/1430599404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f587c102480 msgr2=0x7f587c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f587c102480 0x7f587c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.345+0000 7f5880c7e700 1 -- 172.21.15.47:0/1430599404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587c19a170 con 0x7f587c102e40 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.346+0000 7f5880c7e700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c102e40 0x7f587c19fd90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f586c00c9b0 tx=0x7f586c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.346+0000 7f58727fc700 1 -- 172.21.15.47:0/1430599404 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c007b00 con 0x7f587c102e40 2024-05-11T12:22:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.346+0000 7f58727fc700 1 -- 172.21.15.47:0/1430599404 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f586c00ce50 con 0x7f587c102e40 2024-05-11T12:22:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.346+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587c19a460 con 0x7f587c102e40 2024-05-11T12:22:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.346+0000 7f58727fc700 1 -- 172.21.15.47:0/1430599404 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c018660 con 0x7f587c102e40 2024-05-11T12:22:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.346+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587c1a6720 con 0x7f587c102e40 2024-05-11T12:22:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.347+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f587c06a240 con 0x7f587c102e40 2024-05-11T12:22:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.348+0000 7f58727fc700 1 -- 172.21.15.47:0/1430599404 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f586c00f450 con 0x7f587c102e40 2024-05-11T12:22:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.349+0000 7f58727fc700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f58680b2910 0x7f58680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.349+0000 7f588147f700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f58680b2910 0x7f58680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.349+0000 7f58727fc700 1 -- 172.21.15.47:0/1430599404 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f586c0e25c0 con 0x7f587c102e40 2024-05-11T12:22:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.349+0000 7f588147f700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f58680b2910 0x7f58680b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f587800b5c0 tx=0x7f5878005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.352+0000 7f58727fc700 1 -- 172.21.15.47:0/1430599404 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f586c014030 con 0x7f587c102e40 2024-05-11T12:22:07.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:06 smithi064 ceph-mon[34537]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-05-11T12:22:07.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:06 smithi050 ceph-mon[34072]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-05-11T12:22:07.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.570+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f587c02e570 con 0x7f587c102e40 2024-05-11T12:22:07.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.571+0000 7f58727fc700 1 -- 172.21.15.47:0/1430599404 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22638 (secure 0 0 0) 0x7f586c0aa1f0 con 0x7f587c102e40 2024-05-11T12:22:07.572 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:22:07.572 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":81,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:21:42.658586+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:22:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f58680b2910 msgr2=0x7f58680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f58680b2910 0x7f58680b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f587800b5c0 tx=0x7f5878005960 comp rx=0 tx=0).stop 2024-05-11T12:22:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c102e40 msgr2=0x7f587c19fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c102e40 0x7f587c19fd90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f586c00c9b0 tx=0x7f586c00ccc0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 shutdown_connections 2024-05-11T12:22:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f587c1084c0 0x7f587c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f58680b2910 0x7f58680b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f587c102480 0x7f587c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 --2- 172.21.15.47:0/1430599404 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f587c102e40 0x7f587c19fd90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 >> 172.21.15.47:0/1430599404 conn(0x7f587c079400 msgr2=0x7f587c06bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 shutdown_connections 2024-05-11T12:22:07.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:07.574+0000 7f58836e3700 1 -- 172.21.15.47:0/1430599404 wait complete. 2024-05-11T12:22:08.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:07 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1430599404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:22:08.183 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:07 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:22:08.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-11T12:22:08.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-11T12:22:08.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-11T12:22:08.214 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-11T12:22:08.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-11T12:22:08.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-11T12:22:08.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-11T12:22:08.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-11T12:22:08.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-11T12:22:08.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-11T12:22:08.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-11T12:22:08.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-11T12:22:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:07 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1430599404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:22:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:07 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:22:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:07 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1430599404' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:22:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:07 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:22:09.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:08 smithi047 ceph-mon[30375]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:09.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:08 smithi064 ceph-mon[34537]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:09.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:08 smithi050 ceph-mon[34072]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:09.990 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:09.995 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:09.997 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:10.002 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:10.003 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:10.006 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:10.081 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:10.133 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:10.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:10 smithi047 ceph-mon[30375]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:10.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.845 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:10 smithi064 ceph-mon[34537]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:10 smithi050 ceph-mon[34072]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:10.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:10.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:12.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:12 smithi064 ceph-mon[34537]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:12.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:12 smithi050 ceph-mon[34072]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:13.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:12 smithi047 ceph-mon[30375]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 -- 172.21.15.47:0/1715324809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c1041e0 msgr2=0x7f873c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 --2- 172.21.15.47:0/1715324809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c1041e0 0x7f873c1081f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f87300099c0 tx=0x7f8730009cd0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 -- 172.21.15.47:0/1715324809 shutdown_connections 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 --2- 172.21.15.47:0/1715324809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f873c108920 0x7f873c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 --2- 172.21.15.47:0/1715324809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f873c103830 0x7f873c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 --2- 172.21.15.47:0/1715324809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c1041e0 0x7f873c1081f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 -- 172.21.15.47:0/1715324809 >> 172.21.15.47:0/1715324809 conn(0x7f873c07a6c0 msgr2=0x7f873c07aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 -- 172.21.15.47:0/1715324809 shutdown_connections 2024-05-11T12:22:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.778+0000 7f8740b06700 1 -- 172.21.15.47:0/1715324809 wait complete. 2024-05-11T12:22:13.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 Processor -- start 2024-05-11T12:22:13.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 -- start start 2024-05-11T12:22:13.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f873c103830 0x7f873c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f873c1041e0 0x7f873c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c108920 0x7f873c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f873c1137c0 con 0x7f873c108920 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f873c113640 con 0x7f873c103830 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8740b06700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f873c113940 con 0x7f873c1041e0 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f873ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c108920 0x7f873c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f873ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c108920 0x7f873c199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34650/0 (socket says 172.21.15.47:34650) 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f873ad9d700 1 -- 172.21.15.47:0/3568946641 learned_addr learned my addr 172.21.15.47:0/3568946641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f8739d9b700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f873c1041e0 0x7f873c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.779+0000 7f873a59c700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f873c103830 0x7f873c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f873ad9d700 1 -- 172.21.15.47:0/3568946641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f873c1041e0 msgr2=0x7f873c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f873ad9d700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f873c1041e0 0x7f873c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f873ad9d700 1 -- 172.21.15.47:0/3568946641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f873c103830 msgr2=0x7f873c19f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f873ad9d700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f873c103830 0x7f873c19f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f873ad9d700 1 -- 172.21.15.47:0/3568946641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c19a1c0 con 0x7f873c108920 2024-05-11T12:22:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f873a59c700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f873c103830 0x7f873c19f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:22:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f873ad9d700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c108920 0x7f873c199a60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8734007ee0 tx=0x7f873400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873400dbd0 con 0x7f873c108920 2024-05-11T12:22:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873c19a4b0 con 0x7f873c108920 2024-05-11T12:22:13.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f873400dd70 con 0x7f873c108920 2024-05-11T12:22:13.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734005990 con 0x7f873c108920 2024-05-11T12:22:13.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.780+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873c1a6770 con 0x7f873c108920 2024-05-11T12:22:13.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.781+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f873c063ce0 con 0x7f873c108920 2024-05-11T12:22:13.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.782+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f873401a020 con 0x7f873c108920 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.783+0000 7f872b7fe700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87240b2910 0x7f87240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.783+0000 7f873a59c700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87240b2910 0x7f87240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.783+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f87340e2270 con 0x7f873c108920 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.783+0000 7f872b7fe700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] conn(0x7f87240bc550 0x7f87240be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.783+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 --> [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8734012ec0 con 0x7f87240bc550 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.783+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f87340e26c0 con 0x7f873c108920 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.784+0000 7f8739d9b700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] conn(0x7f87240bc550 0x7f87240be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.784+0000 7f873a59c700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87240b2910 0x7f87240b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f873c19aba0 tx=0x7f872c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.784+0000 7f8739d9b700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] conn(0x7f87240bc550 0x7f87240be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:13.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.785+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== osd.9 v2:172.21.15.47:6826/2171109969 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8734012ec0 con 0x7f87240bc550 2024-05-11T12:22:13.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.801+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 --> [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f873c064160 con 0x7f87240bc550 2024-05-11T12:22:13.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.801+0000 7f872b7fe700 1 -- 172.21.15.47:0/3568946641 <== osd.9 v2:172.21.15.47:6826/2171109969 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f873c064160 con 0x7f87240bc550 2024-05-11T12:22:13.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] conn(0x7f87240bc550 msgr2=0x7f87240be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:13.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] conn(0x7f87240bc550 0x7f87240be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87240b2910 msgr2=0x7f87240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:13.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87240b2910 0x7f87240b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f873c19aba0 tx=0x7f872c008040 comp rx=0 tx=0).stop 2024-05-11T12:22:13.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c108920 msgr2=0x7f873c199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:13.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 --2- 172.21.15.47:0/3568946641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f873c108920 0x7f873c199a60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8734007ee0 tx=0x7f873400d3b0 comp rx=0 tx=0).stop 2024-05-11T12:22:13.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f873ad9d700 1 -- 172.21.15.47:0/3568946641 reap_dead start 2024-05-11T12:22:13.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 shutdown_connections 2024-05-11T12:22:13.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 >> 172.21.15.47:0/3568946641 conn(0x7f873c07a6c0 msgr2=0x7f873c109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:13.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 shutdown_connections 2024-05-11T12:22:13.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:13.802+0000 7f8740b06700 1 -- 172.21.15.47:0/3568946641 wait complete. 2024-05-11T12:22:14.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:14 smithi064 ceph-mon[34537]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:14.905 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:14 smithi047 ceph-mon[30375]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:14.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:14 smithi050 ceph-mon[34072]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:16.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:16 smithi047 ceph-mon[30375]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2347102218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc109740 msgr2=0x7ff8cc109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 --2- 172.21.15.47:0/2347102218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc109740 0x7ff8cc109b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009990 tx=0x7ff8bc009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2347102218 shutdown_connections 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 --2- 172.21.15.47:0/2347102218 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8cc103ff0 0x7ff8cc110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 --2- 172.21.15.47:0/2347102218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc109740 0x7ff8cc109b20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 --2- 172.21.15.47:0/2347102218 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8cc103630 0x7ff8cc103ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2347102218 >> 172.21.15.47:0/2347102218 conn(0x7ff8cc07a620 msgr2=0x7ff8cc07aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:16.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2347102218 shutdown_connections 2024-05-11T12:22:16.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2347102218 wait complete. 2024-05-11T12:22:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.798+0000 7ff8d0e67700 1 Processor -- start 2024-05-11T12:22:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8d0e67700 1 -- start start 2024-05-11T12:22:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8d0e67700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc103630 0x7ff8cc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8d0e67700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8cc103ff0 0x7ff8cc19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:16.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8d0e67700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8cc109740 0x7ff8cc199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8d0e67700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113790 con 0x7ff8cc103630 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8d0e67700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113610 con 0x7ff8cc109740 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8d0e67700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113910 con 0x7ff8cc103ff0 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc103630 0x7ff8cc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc103630 0x7ff8cc19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37934/0 (socket says 172.21.15.47:37934) 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 -- 172.21.15.47:0/2363335598 learned_addr learned my addr 172.21.15.47:0/2363335598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8c9d9b700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8cc103ff0 0x7ff8cc19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8cad9d700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8cc109740 0x7ff8cc199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 -- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8cc103ff0 msgr2=0x7ff8cc19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff8cc103ff0 0x7ff8cc19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 -- 172.21.15.47:0/2363335598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8cc109740 msgr2=0x7ff8cc199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8cc109740 0x7ff8cc199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8ca59c700 1 -- 172.21.15.47:0/2363335598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc19a170 con 0x7ff8cc103630 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.799+0000 7ff8cad9d700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8cc109740 0x7ff8cc199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.800+0000 7ff8ca59c700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc103630 0x7ff8cc19f850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c00 tx=0x7ff8bc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.800+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc027070 con 0x7ff8cc103630 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.800+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc19a400 con 0x7ff8cc103630 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.800+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8bc003f20 con 0x7ff8cc103630 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.800+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc022cf0 con 0x7ff8cc103630 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.800+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1a6720 con 0x7ff8cc103630 2024-05-11T12:22:16.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.800+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff8cc063c60 con 0x7ff8cc103630 2024-05-11T12:22:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.802+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff8bc02a070 con 0x7ff8cc103630 2024-05-11T12:22:16.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.803+0000 7ff8bb7fe700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8b40b2910 0x7ff8b40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.803+0000 7ff8c9d9b700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8b40b2910 0x7ff8b40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.803+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff8bc0e37a0 con 0x7ff8cc103630 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.803+0000 7ff8bb7fe700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] conn(0x7ff8b40bc550 0x7ff8b40be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.803+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 --> [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8bc023e50 con 0x7ff8b40bc550 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.803+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ff8bc0e3bf0 con 0x7ff8cc103630 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.803+0000 7ff8c9d9b700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8b40b2910 0x7ff8b40b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc108d70 tx=0x7ff8c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.804+0000 7ff8cad9d700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] conn(0x7ff8b40bc550 0x7ff8b40be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.804+0000 7ff8cad9d700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] conn(0x7ff8b40bc550 0x7ff8b40be9f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:16.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.805+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== osd.11 v2:172.21.15.64:6824/368350106 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff8bc023e50 con 0x7ff8b40bc550 2024-05-11T12:22:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.822+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 --> [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff8cc064060 con 0x7ff8b40bc550 2024-05-11T12:22:16.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8bb7fe700 1 -- 172.21.15.47:0/2363335598 <== osd.11 v2:172.21.15.64:6824/368350106 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff8cc064060 con 0x7ff8b40bc550 2024-05-11T12:22:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] conn(0x7ff8b40bc550 msgr2=0x7ff8b40be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] conn(0x7ff8b40bc550 0x7ff8b40be9f0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8b40b2910 msgr2=0x7ff8b40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:16.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8b40b2910 0x7ff8b40b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc108d70 tx=0x7ff8c000b410 comp rx=0 tx=0).stop 2024-05-11T12:22:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc103630 msgr2=0x7ff8cc19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 --2- 172.21.15.47:0/2363335598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc103630 0x7ff8cc19f850 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c00 tx=0x7ff8bc0047b0 comp rx=0 tx=0).stop 2024-05-11T12:22:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8cad9d700 1 -- 172.21.15.47:0/2363335598 reap_dead start 2024-05-11T12:22:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 shutdown_connections 2024-05-11T12:22:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 >> 172.21.15.47:0/2363335598 conn(0x7ff8cc07a620 msgr2=0x7ff8cc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 shutdown_connections 2024-05-11T12:22:16.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:16.823+0000 7ff8d0e67700 1 -- 172.21.15.47:0/2363335598 wait complete. 2024-05-11T12:22:16.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:16 smithi064 ceph-mon[34537]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:16.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:16 smithi050 ceph-mon[34072]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:18.046 INFO:teuthology.orchestra.run.smithi047.stdout:257698037799 2024-05-11T12:22:18.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-11T12:22:18.824 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:18 smithi047 ceph-mon[30375]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:18.853 INFO:teuthology.orchestra.run.smithi047.stdout:304942678046 2024-05-11T12:22:18.854 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-11T12:22:18.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:18 smithi064 ceph-mon[34537]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:18.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:18 smithi050 ceph-mon[34072]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:19.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 -- 172.21.15.47:0/3160852835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 msgr2=0x7fc44c110410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 --2- 172.21.15.47:0/3160852835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 0x7fc44c110410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc44400b0a0 tx=0x7fc44400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 -- 172.21.15.47:0/3160852835 shutdown_connections 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 --2- 172.21.15.47:0/3160852835 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 0x7fc44c110410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 --2- 172.21.15.47:0/3160852835 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc44c1095f0 0x7fc44c1099d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 --2- 172.21.15.47:0/3160852835 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc44c06bc20 0x7fc44c06c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 -- 172.21.15.47:0/3160852835 >> 172.21.15.47:0/3160852835 conn(0x7fc44c079490 msgr2=0x7fc44c0798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.043+0000 7fc450ed9700 1 -- 172.21.15.47:0/3160852835 shutdown_connections 2024-05-11T12:22:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.044+0000 7fc450ed9700 1 -- 172.21.15.47:0/3160852835 wait complete. 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.044+0000 7fc450ed9700 1 Processor -- start 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.044+0000 7fc450ed9700 1 -- start start 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc450ed9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc44c06bc20 0x7fc44c19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc450ed9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 0x7fc44c19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc450ed9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc44c1095f0 0x7fc44c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc450ed9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c113780 con 0x7fc44c06c5e0 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc450ed9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c113600 con 0x7fc44c1095f0 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc450ed9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc44c113900 con 0x7fc44c06bc20 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 0x7fc44c19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 0x7fc44c19fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37956/0 (socket says 172.21.15.47:37956) 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc44a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc44c06bc20 0x7fc44c19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 -- 172.21.15.47:0/3150743700 learned_addr learned my addr 172.21.15.47:0/3150743700 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc44ad9d700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc44c1095f0 0x7fc44c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 -- 172.21.15.47:0/3150743700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc44c06bc20 msgr2=0x7fc44c19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc44c06bc20 0x7fc44c19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 -- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc44c1095f0 msgr2=0x7fc44c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc44c1095f0 0x7fc44c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc449d9b700 1 -- 172.21.15.47:0/3150743700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44c19a160 con 0x7fc44c06c5e0 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.045+0000 7fc44a59c700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc44c06bc20 0x7fc44c19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.046+0000 7fc449d9b700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 0x7fc44c19fd80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc44000d980 tx=0x7fc44000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.046+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440016070 con 0x7fc44c06c5e0 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.046+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44c19a450 con 0x7fc44c06c5e0 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.046+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc440004830 con 0x7fc44c06c5e0 2024-05-11T12:22:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.046+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440005540 con 0x7fc44c06c5e0 2024-05-11T12:22:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.046+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44c102480 con 0x7fc44c06c5e0 2024-05-11T12:22:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.047+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc44c063ce0 con 0x7fc44c06c5e0 2024-05-11T12:22:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.048+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc440004220 con 0x7fc44c06c5e0 2024-05-11T12:22:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.049+0000 7fc43b7fe700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc4340b2990 0x7fc4340b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.049+0000 7fc44a59c700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc4340b2990 0x7fc4340b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.049+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fc4400e27c0 con 0x7fc44c06c5e0 2024-05-11T12:22:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.049+0000 7fc43b7fe700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] conn(0x7fc4340bc5d0 0x7fc4340bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.049+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 --> [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc44001ce50 con 0x7fc4340bc5d0 2024-05-11T12:22:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.049+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc4400e2c10 con 0x7fc44c06c5e0 2024-05-11T12:22:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.050+0000 7fc44a59c700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc4340b2990 0x7fc4340b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005950 tx=0x7fc43c010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.050+0000 7fc44ad9d700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] conn(0x7fc4340bc5d0 0x7fc4340bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.050+0000 7fc44ad9d700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] conn(0x7fc4340bc5d0 0x7fc4340bea70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.051+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== osd.7 v2:172.21.15.50:6816/1770681841 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc44001ce50 con 0x7fc4340bc5d0 2024-05-11T12:22:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.069+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 --> [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc44c0640e0 con 0x7fc4340bc5d0 2024-05-11T12:22:19.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc43b7fe700 1 -- 172.21.15.47:0/3150743700 <== osd.7 v2:172.21.15.50:6816/1770681841 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc44c0640e0 con 0x7fc4340bc5d0 2024-05-11T12:22:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] conn(0x7fc4340bc5d0 msgr2=0x7fc4340bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] conn(0x7fc4340bc5d0 0x7fc4340bea70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc4340b2990 msgr2=0x7fc4340b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc4340b2990 0x7fc4340b4e50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc43c005950 tx=0x7fc43c010040 comp rx=0 tx=0).stop 2024-05-11T12:22:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 msgr2=0x7fc44c19fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 --2- 172.21.15.47:0/3150743700 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc44c06c5e0 0x7fc44c19fd80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc44000d980 tx=0x7fc44000dc90 comp rx=0 tx=0).stop 2024-05-11T12:22:19.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc44ad9d700 1 -- 172.21.15.47:0/3150743700 reap_dead start 2024-05-11T12:22:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 shutdown_connections 2024-05-11T12:22:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 >> 172.21.15.47:0/3150743700 conn(0x7fc44c079490 msgr2=0x7fc44c101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.070+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 shutdown_connections 2024-05-11T12:22:19.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.071+0000 7fc450ed9700 1 -- 172.21.15.47:0/3150743700 wait complete. 2024-05-11T12:22:19.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 -- 172.21.15.47:0/3276011366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314108930 msgr2=0x7f83141104b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 --2- 172.21.15.47:0/3276011366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314108930 0x7f83141104b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f830c00b0d0 tx=0x7f830c00b3e0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 -- 172.21.15.47:0/3276011366 shutdown_connections 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 --2- 172.21.15.47:0/3276011366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314108930 0x7f83141104b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 --2- 172.21.15.47:0/3276011366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314103860 0x7f8314103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 --2- 172.21.15.47:0/3276011366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8314104210 0x7f8314108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 -- 172.21.15.47:0/3276011366 >> 172.21.15.47:0/3276011366 conn(0x7f831407a5e0 msgr2=0x7f831407a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 -- 172.21.15.47:0/3276011366 shutdown_connections 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.577+0000 7f8319cac700 1 -- 172.21.15.47:0/3276011366 wait complete. 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 Processor -- start 2024-05-11T12:22:19.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 -- start start 2024-05-11T12:22:19.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314103860 0x7f8314077440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314104210 0x7f8314075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8314108930 0x7f8314075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8314113720 con 0x7f8314104210 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f83141135a0 con 0x7f8314108930 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f8319cac700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f83141138a0 con 0x7f8314103860 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f830affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314104210 0x7f8314075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f830affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314104210 0x7f8314075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37970/0 (socket says 172.21.15.47:37970) 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.578+0000 7f830affd700 1 -- 172.21.15.47:0/2978433321 learned_addr learned my addr 172.21.15.47:0/2978433321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f8313fff700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8314108930 0x7f8314075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f83137fe700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314103860 0x7f8314077440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f830affd700 1 -- 172.21.15.47:0/2978433321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314103860 msgr2=0x7f8314077440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f830affd700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8314103860 0x7f8314077440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f830affd700 1 -- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8314108930 msgr2=0x7f8314075eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f830affd700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8314108930 0x7f8314075eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f830affd700 1 -- 172.21.15.47:0/2978433321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83141ac580 con 0x7f8314104210 2024-05-11T12:22:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f8313fff700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8314108930 0x7f8314075eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:22:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f830affd700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314104210 0x7f8314075850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00d950 tx=0x7f82fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc016070 con 0x7f8314104210 2024-05-11T12:22:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.579+0000 7f8319cac700 1 -- 172.21.15.47:0/2978433321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83141ac810 con 0x7f8314104210 2024-05-11T12:22:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.580+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82fc009df0 con 0x7f8314104210 2024-05-11T12:22:19.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.580+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc00f3c0 con 0x7f8314104210 2024-05-11T12:22:19.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.580+0000 7f8319cac700 1 -- 172.21.15.47:0/2978433321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83141acd50 con 0x7f8314104210 2024-05-11T12:22:19.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.581+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8314063c60 con 0x7f8314104210 2024-05-11T12:22:19.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.583+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f82fc019070 con 0x7f8314104210 2024-05-11T12:22:19.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.585+0000 7f83117fa700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f83000b2910 0x7f83000b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.585+0000 7f83137fe700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f83000b2910 0x7f83000b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.585+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f82fc0e2120 con 0x7f8314104210 2024-05-11T12:22:19.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.585+0000 7f83117fa700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] conn(0x7f83000bc550 0x7f83000be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.585+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 --> [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83000bf0c0 con 0x7f83000bc550 2024-05-11T12:22:19.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.585+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f82fc0e2570 con 0x7f8314104210 2024-05-11T12:22:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.585+0000 7f83137fe700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f83000b2910 0x7f83000b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8304009bf0 tx=0x7f8304009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.586+0000 7f8313fff700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] conn(0x7f83000bc550 0x7f83000be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.586+0000 7f8313fff700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] conn(0x7f83000bc550 0x7f83000be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.587+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== osd.10 v2:172.21.15.50:6824/1444033681 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83000bf0c0 con 0x7f83000bc550 2024-05-11T12:22:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.604+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 --> [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83140640e0 con 0x7f83000bc550 2024-05-11T12:22:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f83117fa700 1 -- 172.21.15.47:0/2978433321 <== osd.10 v2:172.21.15.50:6824/1444033681 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f83140640e0 con 0x7f83000bc550 2024-05-11T12:22:19.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] conn(0x7f83000bc550 msgr2=0x7f83000be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f830a7fc700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] conn(0x7f83000bc550 0x7f83000be9f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f83000b2910 msgr2=0x7f83000b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f830a7fc700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f83000b2910 0x7f83000b4dd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8304009bf0 tx=0x7f8304009260 comp rx=0 tx=0).stop 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314104210 msgr2=0x7f8314075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f830a7fc700 1 --2- 172.21.15.47:0/2978433321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8314104210 0x7f8314075850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00d950 tx=0x7f82fc00dc60 comp rx=0 tx=0).stop 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.605+0000 7f8313fff700 1 -- 172.21.15.47:0/2978433321 reap_dead start 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.606+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 shutdown_connections 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.606+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 >> 172.21.15.47:0/2978433321 conn(0x7f831407a5e0 msgr2=0x7f8314101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.606+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 shutdown_connections 2024-05-11T12:22:19.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:19.606+0000 7f830a7fc700 1 -- 172.21.15.47:0/2978433321 wait complete. 2024-05-11T12:22:19.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.655+0000 7f023002e700 1 -- 172.21.15.47:0/1081232007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 msgr2=0x7f0228108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.655+0000 7f023002e700 1 --2- 172.21.15.47:0/1081232007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 0x7f0228108900 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f022400b0a0 tx=0x7f022400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.655+0000 7f023002e700 1 -- 172.21.15.47:0/1081232007 shutdown_connections 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.655+0000 7f023002e700 1 --2- 172.21.15.47:0/1081232007 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0228102e00 0x7f022810f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.655+0000 7f023002e700 1 --2- 172.21.15.47:0/1081232007 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 0x7f0228108900 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.655+0000 7f023002e700 1 --2- 172.21.15.47:0/1081232007 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102440 0x7f02281028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.655+0000 7f023002e700 1 -- 172.21.15.47:0/1081232007 >> 172.21.15.47:0/1081232007 conn(0x7f02280793c0 msgr2=0x7f02280797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.656+0000 7f023002e700 1 -- 172.21.15.47:0/1081232007 shutdown_connections 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.656+0000 7f023002e700 1 -- 172.21.15.47:0/1081232007 wait complete. 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.656+0000 7f023002e700 1 Processor -- start 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.656+0000 7f023002e700 1 -- start start 2024-05-11T12:22:19.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f023002e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102440 0x7f02281040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f023002e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0228102e00 0x7f02281045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f023002e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 0x7f0228104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f023002e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f02281125d0 con 0x7f0228108520 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f023002e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0228112450 con 0x7f0228102e00 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f023002e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0228112750 con 0x7f0228102440 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 0x7f0228104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 0x7f0228104b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34672/0 (socket says 172.21.15.47:34672) 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 -- 172.21.15.47:0/1466167822 learned_addr learned my addr 172.21.15.47:0/1466167822 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022d5c9700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0228102e00 0x7f02281045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 -- 172.21.15.47:0/1466167822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102440 msgr2=0x7f02281040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0228102440 0x7f02281040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 -- 172.21.15.47:0/1466167822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0228102e00 msgr2=0x7f02281045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0228102e00 0x7f02281045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 -- 172.21.15.47:0/1466167822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02281ac5a0 con 0x7f0228108520 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f022e5cb700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 0x7f0228104b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f022000d980 tx=0x7f022000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0220016070 con 0x7f0228108520 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.657+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0220004830 con 0x7f0228108520 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.658+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02281ac890 con 0x7f0228108520 2024-05-11T12:22:19.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.658+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02200054b0 con 0x7f0228108520 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.658+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02281acda0 con 0x7f0228108520 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.659+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f02281074a0 con 0x7f0228108520 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.660+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f02200049e0 con 0x7f0228108520 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.661+0000 7f021effd700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f02140b2990 0x7f02140b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.661+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f02200e2160 con 0x7f0228108520 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.661+0000 7f022ddca700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f02140b2990 0x7f02140b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.661+0000 7f021effd700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] conn(0x7f02140bc5d0 0x7f02140bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.661+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 --> [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02140bf140 con 0x7f02140bc5d0 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.661+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f02200e9050 con 0x7f0228108520 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.662+0000 7f022d5c9700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] conn(0x7f02140bc5d0 0x7f02140bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.662+0000 7f022ddca700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f02140b2990 0x7f02140b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f022400b070 tx=0x7f02240060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.662+0000 7f022d5c9700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] conn(0x7f02140bc5d0 0x7f02140bea70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.663+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== osd.5 v2:172.21.15.64:6808/1906914370 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f02140bf140 con 0x7f02140bc5d0 2024-05-11T12:22:19.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.680+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 --> [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0228103b60 con 0x7f02140bc5d0 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.680+0000 7f021effd700 1 -- 172.21.15.47:0/1466167822 <== osd.5 v2:172.21.15.64:6808/1906914370 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0228103b60 con 0x7f02140bc5d0 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 >> [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] conn(0x7f02140bc5d0 msgr2=0x7f02140bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] conn(0x7f02140bc5d0 0x7f02140bea70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f02140b2990 msgr2=0x7f02140b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f02140b2990 0x7f02140b4e50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f022400b070 tx=0x7f02240060c0 comp rx=0 tx=0).stop 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 msgr2=0x7f0228104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 --2- 172.21.15.47:0/1466167822 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0228108520 0x7f0228104b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f022000d980 tx=0x7f022000dc90 comp rx=0 tx=0).stop 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f022e5cb700 1 -- 172.21.15.47:0/1466167822 reap_dead start 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 shutdown_connections 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 >> 172.21.15.47:0/1466167822 conn(0x7f02280793c0 msgr2=0x7f022806bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 shutdown_connections 2024-05-11T12:22:19.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:15.681+0000 7f023002e700 1 -- 172.21.15.47:0/1466167822 wait complete. 2024-05-11T12:22:20.358 INFO:teuthology.orchestra.run.smithi047.stdout:158913790012 2024-05-11T12:22:20.358 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-11T12:22:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:20 smithi064 ceph-mon[34537]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:20.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:20 smithi050 ceph-mon[34072]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:20.949 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:20 smithi047 ceph-mon[30375]: pgmap v330: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:20.976 INFO:teuthology.orchestra.run.smithi047.stdout:279172874274 2024-05-11T12:22:20.976 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-11T12:22:21.112 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:22.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:22 smithi050 ceph-mon[34072]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:22.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:22:22.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:22:22.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:22 smithi047 ceph-mon[30375]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:22:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:22:22.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:22 smithi064 ceph-mon[34537]: pgmap v331: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:22.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:22:22.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:22:22.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:22:22.845 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:23.137 INFO:teuthology.orchestra.run.smithi047.stdout:206158430259 2024-05-11T12:22:23.137 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-11T12:22:23.569 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:24.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:24 smithi064 ceph-mon[34537]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:24.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:24 smithi047 ceph-mon[30375]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:24.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:24 smithi050 ceph-mon[34072]: pgmap v332: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:25.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/3583778769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 msgr2=0x7fd2cc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:25.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 --2- 172.21.15.47:0/3583778769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 0x7fd2cc108900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009990 tx=0x7fd2bc009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/3583778769 shutdown_connections 2024-05-11T12:22:25.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 --2- 172.21.15.47:0/3583778769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2cc102e00 0x7fd2cc10f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 --2- 172.21.15.47:0/3583778769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 0x7fd2cc108900 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 --2- 172.21.15.47:0/3583778769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc102440 0x7fd2cc1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/3583778769 >> 172.21.15.47:0/3583778769 conn(0x7fd2cc0793c0 msgr2=0x7fd2cc0797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/3583778769 shutdown_connections 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.835+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/3583778769 wait complete. 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 Processor -- start 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 -- start start 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc102440 0x7fd2cc19fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2cc102e00 0x7fd2cc19ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 0x7fd2cc199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc1125f0 con 0x7fd2cc108520 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc112470 con 0x7fd2cc102e00 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2d1f0c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc112770 con 0x7fd2cc102440 2024-05-11T12:22:25.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2cbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 0x7fd2cc199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2cbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 0x7fd2cc199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37974/0 (socket says 172.21.15.47:37974) 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2cbfff700 1 -- 172.21.15.47:0/2441103294 learned_addr learned my addr 172.21.15.47:0/2441103294 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2caffd700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2cc102e00 0x7fd2cc19ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.836+0000 7fd2cb7fe700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc102440 0x7fd2cc19fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2cbfff700 1 -- 172.21.15.47:0/2441103294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc102440 msgr2=0x7fd2cc19fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2cbfff700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2cc102440 0x7fd2cc19fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2cbfff700 1 -- 172.21.15.47:0/2441103294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2cc102e00 msgr2=0x7fd2cc19ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2cbfff700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2cc102e00 0x7fd2cc19ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2cbfff700 1 -- 172.21.15.47:0/2441103294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc19a440 con 0x7fd2cc108520 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2cbfff700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 0x7fd2cc199b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400ef30 tx=0x7fd2c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c40092e0 con 0x7fd2cc108520 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c4004bb0 con 0x7fd2cc108520 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc19a6d0 con 0x7fd2cc108520 2024-05-11T12:22:25.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4004280 con 0x7fd2cc108520 2024-05-11T12:22:25.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.837+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc1a67f0 con 0x7fd2cc108520 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.838+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2cc107420 con 0x7fd2cc108520 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.840+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd2c401b030 con 0x7fd2cc108520 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2c8ff9700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2b40b2910 0x7fd2b40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2cb7fe700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2b40b2910 0x7fd2b40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd2c40e2840 con 0x7fd2cc108520 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2c8ff9700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] conn(0x7fd2b40bc550 0x7fd2b40be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 --> [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2b40bf0c0 con 0x7fd2b40bc550 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fd2c40e2c90 con 0x7fd2cc108520 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2caffd700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] conn(0x7fd2b40bc550 0x7fd2b40be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:25.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.841+0000 7fd2cb7fe700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2b40b2910 0x7fd2b40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc006010 tx=0x7fd2bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:25.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.842+0000 7fd2caffd700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] conn(0x7fd2b40bc550 0x7fd2b40be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:25.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.842+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== osd.6 v2:172.21.15.47:6818/3223062314 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2b40bf0c0 con 0x7fd2b40bc550 2024-05-11T12:22:25.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 --> [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2cc103b60 con 0x7fd2b40bc550 2024-05-11T12:22:25.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2c8ff9700 1 -- 172.21.15.47:0/2441103294 <== osd.6 v2:172.21.15.47:6818/3223062314 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd2cc103b60 con 0x7fd2b40bc550 2024-05-11T12:22:25.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] conn(0x7fd2b40bc550 msgr2=0x7fd2b40be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:25.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] conn(0x7fd2b40bc550 0x7fd2b40be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2b40b2910 msgr2=0x7fd2b40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2b40b2910 0x7fd2b40b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc006010 tx=0x7fd2bc01a040 comp rx=0 tx=0).stop 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 msgr2=0x7fd2cc199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 --2- 172.21.15.47:0/2441103294 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2cc108520 0x7fd2cc199b60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400ef30 tx=0x7fd2c400c3a0 comp rx=0 tx=0).stop 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2cbfff700 1 -- 172.21.15.47:0/2441103294 reap_dead start 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 shutdown_connections 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.861+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 >> 172.21.15.47:0/2441103294 conn(0x7fd2cc0793c0 msgr2=0x7fd2cc06bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.862+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 shutdown_connections 2024-05-11T12:22:25.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:25.862+0000 7fd2d1f0c700 1 -- 172.21.15.47:0/2441103294 wait complete. 2024-05-11T12:22:26.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:26 smithi064 ceph-mon[34537]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:26.894 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:26 smithi047 ceph-mon[30375]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:26.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:26 smithi050 ceph-mon[34072]: pgmap v333: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.998+0000 7fd2aceec700 1 -- 172.21.15.47:0/3301132303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8108540 msgr2=0x7fd2a8108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.998+0000 7fd2aceec700 1 --2- 172.21.15.47:0/3301132303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8108540 0x7fd2a8108920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd2980099c0 tx=0x7fd298009cd0 comp rx=0 tx=0).stop 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 -- 172.21.15.47:0/3301132303 shutdown_connections 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 --2- 172.21.15.47:0/3301132303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a8102e50 0x7fd2a810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 --2- 172.21.15.47:0/3301132303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8108540 0x7fd2a8108920 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 --2- 172.21.15.47:0/3301132303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2a8102490 0x7fd2a8102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 -- 172.21.15.47:0/3301132303 >> 172.21.15.47:0/3301132303 conn(0x7fd2a80794f0 msgr2=0x7fd2a8079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 -- 172.21.15.47:0/3301132303 shutdown_connections 2024-05-11T12:22:26.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 -- 172.21.15.47:0/3301132303 wait complete. 2024-05-11T12:22:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 Processor -- start 2024-05-11T12:22:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:26.999+0000 7fd2aceec700 1 -- start start 2024-05-11T12:22:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2aceec700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8102490 0x7fd2a8104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2aceec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2a8102e50 0x7fd2a8104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2aceec700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a8108540 0x7fd2a8104b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:27.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2aceec700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a81126a0 con 0x7fd2a8102490 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2aceec700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8112520 con 0x7fd2a8102e50 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2aceec700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8112820 con 0x7fd2a8108540 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8102490 0x7fd2a8104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8102490 0x7fd2a8104100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34506/0 (socket says 172.21.15.47:34506) 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 -- 172.21.15.47:0/273825213 learned_addr learned my addr 172.21.15.47:0/273825213 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a6d9d700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a8108540 0x7fd2a8104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a5d9b700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2a8102e50 0x7fd2a8104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 -- 172.21.15.47:0/273825213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a8108540 msgr2=0x7fd2a8104b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd2a8108540 0x7fd2a8104b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 -- 172.21.15.47:0/273825213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2a8102e50 msgr2=0x7fd2a8104640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd2a8102e50 0x7fd2a8104640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 -- 172.21.15.47:0/273825213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81a8110 con 0x7fd2a8102490 2024-05-11T12:22:27.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.000+0000 7fd2a659c700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8102490 0x7fd2a8104100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd29800b580 tx=0x7fd2980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.001+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298027070 con 0x7fd2a8102490 2024-05-11T12:22:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.001+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd298003f20 con 0x7fd2a8102490 2024-05-11T12:22:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.001+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a81a83a0 con 0x7fd2a8102490 2024-05-11T12:22:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.001+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd298022d10 con 0x7fd2a8102490 2024-05-11T12:22:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.001+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a8850 con 0x7fd2a8102490 2024-05-11T12:22:27.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.001+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd2a8106590 con 0x7fd2a8102490 2024-05-11T12:22:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.003+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd298022470 con 0x7fd2a8102490 2024-05-11T12:22:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2977fe700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2900b2990 0x7fd2900b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2a5d9b700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2900b2990 0x7fd2900b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd2980e3520 con 0x7fd2a8102490 2024-05-11T12:22:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2977fe700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] conn(0x7fd2900bc4a0 0x7fd2900be940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 --> [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd2980043d0 con 0x7fd2900bc4a0 2024-05-11T12:22:27.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fd2980e3970 con 0x7fd2a8102490 2024-05-11T12:22:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2a6d9d700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] conn(0x7fd2900bc4a0 0x7fd2900be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.004+0000 7fd2a5d9b700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2900b2990 0x7fd2900b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8079a60 tx=0x7fd29c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.005+0000 7fd2a6d9d700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] conn(0x7fd2900bc4a0 0x7fd2900be940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:27.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.005+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== osd.3 v2:172.21.15.47:6810/425091101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd2980043d0 con 0x7fd2900bc4a0 2024-05-11T12:22:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.022+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 --> [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd2a8103bb0 con 0x7fd2900bc4a0 2024-05-11T12:22:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2977fe700 1 -- 172.21.15.47:0/273825213 <== osd.3 v2:172.21.15.47:6810/425091101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd2a8103bb0 con 0x7fd2900bc4a0 2024-05-11T12:22:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] conn(0x7fd2900bc4a0 msgr2=0x7fd2900be940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] conn(0x7fd2900bc4a0 0x7fd2900be940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2900b2990 msgr2=0x7fd2900b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd2900b2990 0x7fd2900b4e50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8079a60 tx=0x7fd29c009190 comp rx=0 tx=0).stop 2024-05-11T12:22:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8102490 msgr2=0x7fd2a8104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 --2- 172.21.15.47:0/273825213 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd2a8102490 0x7fd2a8104100 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd29800b580 tx=0x7fd2980047b0 comp rx=0 tx=0).stop 2024-05-11T12:22:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2a6d9d700 1 -- 172.21.15.47:0/273825213 reap_dead start 2024-05-11T12:22:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 shutdown_connections 2024-05-11T12:22:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 >> 172.21.15.47:0/273825213 conn(0x7fd2a80794f0 msgr2=0x7fd2a8101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 shutdown_connections 2024-05-11T12:22:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:27.023+0000 7fd2aceec700 1 -- 172.21.15.47:0/273825213 wait complete. 2024-05-11T12:22:27.873 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 -- 172.21.15.47:0/1732631405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730103800 msgr2=0x7f0730103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 --2- 172.21.15.47:0/1732631405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730103800 0x7f0730103be0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f07200099c0 tx=0x7f0720009cd0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 -- 172.21.15.47:0/1732631405 shutdown_connections 2024-05-11T12:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 --2- 172.21.15.47:0/1732631405 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0730108900 0x7f0730110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 --2- 172.21.15.47:0/1732631405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730103800 0x7f0730103be0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 --2- 172.21.15.47:0/1732631405 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f07301041b0 0x7f07301081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 -- 172.21.15.47:0/1732631405 >> 172.21.15.47:0/1732631405 conn(0x7f073007a6b0 msgr2=0x7f073007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 -- 172.21.15.47:0/1732631405 shutdown_connections 2024-05-11T12:22:28.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.074+0000 7f07362eb700 1 -- 172.21.15.47:0/1732631405 wait complete. 2024-05-11T12:22:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.075+0000 7f07362eb700 1 Processor -- start 2024-05-11T12:22:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.075+0000 7f07362eb700 1 -- start start 2024-05-11T12:22:28.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.075+0000 7f07362eb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103800 0x7f073019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.075+0000 7f07362eb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07301041b0 0x7f073019fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f07362eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730108900 0x7f0730199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f07362eb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0730113700 con 0x7f0730108900 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f07362eb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0730113580 con 0x7f07301041b0 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f07362eb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0730113880 con 0x7f0730103800 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730108900 0x7f0730199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730108900 0x7f0730199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34518/0 (socket says 172.21.15.47:34518) 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 -- 172.21.15.47:0/2211652281 learned_addr learned my addr 172.21.15.47:0/2211652281 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f072ffff700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103800 0x7f073019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f072f7fe700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07301041b0 0x7f073019fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 -- 172.21.15.47:0/2211652281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103800 msgr2=0x7f073019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103800 0x7f073019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 -- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07301041b0 msgr2=0x7f073019fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07301041b0 0x7f073019fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 -- 172.21.15.47:0/2211652281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073019a1e0 con 0x7f0730108900 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f072f7fe700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f07301041b0 0x7f073019fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f072ffff700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0730103800 0x7f073019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f0734888700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730108900 0x7f0730199a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f072800ada0 tx=0x7f072800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728017020 con 0x7f0730108900 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073019a4d0 con 0x7f0730108900 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f072800ce30 con 0x7f0730108900 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07280054a0 con 0x7f0730108900 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.076+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07301a6790 con 0x7f0730108900 2024-05-11T12:22:28.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.077+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0730063ca0 con 0x7f0730108900 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.079+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f072800c980 con 0x7f0730108900 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.080+0000 7f072d7fa700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07180b2910 0x7f07180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.080+0000 7f072ffff700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07180b2910 0x7f07180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.080+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f07280e25e0 con 0x7f0730108900 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.080+0000 7f072d7fa700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] conn(0x7f07180bc550 0x7f07180be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.080+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 --> [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0728022e50 con 0x7f07180bc550 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.080+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f07280e2a30 con 0x7f0730108900 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.081+0000 7f072ffff700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07180b2910 0x7f07180b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0720000c00 tx=0x7f0720005940 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.081+0000 7f072f7fe700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] conn(0x7f07180bc550 0x7f07180be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.081+0000 7f072f7fe700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] conn(0x7f07180bc550 0x7f07180be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:28.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.082+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== osd.4 v2:172.21.15.50:6808/3426956641 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0728022e50 con 0x7f07180bc550 2024-05-11T12:22:28.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.102+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 --> [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0730064150 con 0x7f07180bc550 2024-05-11T12:22:28.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.102+0000 7f072d7fa700 1 -- 172.21.15.47:0/2211652281 <== osd.4 v2:172.21.15.50:6808/3426956641 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0730064150 con 0x7f07180bc550 2024-05-11T12:22:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] conn(0x7f07180bc550 msgr2=0x7f07180be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] conn(0x7f07180bc550 0x7f07180be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07180b2910 msgr2=0x7f07180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07180b2910 0x7f07180b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0720000c00 tx=0x7f0720005940 comp rx=0 tx=0).stop 2024-05-11T12:22:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730108900 msgr2=0x7f0730199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 --2- 172.21.15.47:0/2211652281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0730108900 0x7f0730199a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f072800ada0 tx=0x7f072800c330 comp rx=0 tx=0).stop 2024-05-11T12:22:28.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f0734888700 1 -- 172.21.15.47:0/2211652281 reap_dead start 2024-05-11T12:22:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 shutdown_connections 2024-05-11T12:22:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 >> 172.21.15.47:0/2211652281 conn(0x7f073007a6b0 msgr2=0x7f0730101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 shutdown_connections 2024-05-11T12:22:28.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.103+0000 7f07362eb700 1 -- 172.21.15.47:0/2211652281 wait complete. 2024-05-11T12:22:28.867 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:28 smithi047 ceph-mon[30375]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.899+0000 7f5672677700 1 -- 172.21.15.47:0/3416733883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c1041e0 msgr2=0x7f566c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.899+0000 7f5672677700 1 --2- 172.21.15.47:0/3416733883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c1041e0 0x7f566c1081f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f56600099c0 tx=0x7f5660009cd0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.899+0000 7f5672677700 1 -- 172.21.15.47:0/3416733883 shutdown_connections 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.899+0000 7f5672677700 1 --2- 172.21.15.47:0/3416733883 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566c108920 0x7f566c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.899+0000 7f5672677700 1 --2- 172.21.15.47:0/3416733883 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f566c103830 0x7f566c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.899+0000 7f5672677700 1 --2- 172.21.15.47:0/3416733883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c1041e0 0x7f566c1081f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.899+0000 7f5672677700 1 -- 172.21.15.47:0/3416733883 >> 172.21.15.47:0/3416733883 conn(0x7f566c07a6c0 msgr2=0x7f566c07aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.900+0000 7f5672677700 1 -- 172.21.15.47:0/3416733883 shutdown_connections 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.900+0000 7f5672677700 1 -- 172.21.15.47:0/3416733883 wait complete. 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.900+0000 7f5672677700 1 Processor -- start 2024-05-11T12:22:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.900+0000 7f5672677700 1 -- start start 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f5672677700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c103830 0x7f566c19f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f5672677700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f566c1041e0 0x7f566c19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f5672677700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566c108920 0x7f566c199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f5672677700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f566c1137e0 con 0x7f566c103830 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f5672677700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f566c113660 con 0x7f566c1041e0 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f5672677700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f566c113960 con 0x7f566c108920 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c103830 0x7f566c19f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c103830 0x7f566c19f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34548/0 (socket says 172.21.15.47:34548) 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 -- 172.21.15.47:0/3069219731 learned_addr learned my addr 172.21.15.47:0/3069219731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f5670c14700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566c108920 0x7f566c199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566b7fe700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f566c1041e0 0x7f566c19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 -- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566c108920 msgr2=0x7f566c199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f566c108920 0x7f566c199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 -- 172.21.15.47:0/3069219731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f566c1041e0 msgr2=0x7f566c19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f566c1041e0 0x7f566c19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 -- 172.21.15.47:0/3069219731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c19a1a0 con 0x7f566c103830 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f566bfff700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c103830 0x7f566c19f880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b530 tx=0x7f565c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c015020 con 0x7f566c103830 2024-05-11T12:22:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.901+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c011ce0 con 0x7f566c103830 2024-05-11T12:22:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.902+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c19a490 con 0x7f566c103830 2024-05-11T12:22:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.902+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c010dd0 con 0x7f566c103830 2024-05-11T12:22:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.902+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1a6750 con 0x7f566c103830 2024-05-11T12:22:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.902+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f566c063ce0 con 0x7f566c103830 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.904+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f565c018020 con 0x7f566c103830 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f56697fa700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f56540b2910 0x7f56540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f566b7fe700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f56540b2910 0x7f56540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f565c0e2000 con 0x7f566c103830 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f56697fa700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] conn(0x7f56540bc550 0x7f56540be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 --> [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f565c01be50 con 0x7f56540bc550 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f565c0e24e0 con 0x7f566c103830 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f566b7fe700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f56540b2910 0x7f56540b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5660009990 tx=0x7f56600096d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.905+0000 7f5670c14700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] conn(0x7f56540bc550 0x7f56540be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.906+0000 7f5670c14700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] conn(0x7f56540bc550 0x7f56540be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.907+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== osd.8 v2:172.21.15.64:6816/2722012694 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f565c01be50 con 0x7f56540bc550 2024-05-11T12:22:28.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.924+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 --> [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f566c064160 con 0x7f56540bc550 2024-05-11T12:22:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.924+0000 7f56697fa700 1 -- 172.21.15.47:0/3069219731 <== osd.8 v2:172.21.15.64:6816/2722012694 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f566c064160 con 0x7f56540bc550 2024-05-11T12:22:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] conn(0x7f56540bc550 msgr2=0x7f56540be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] conn(0x7f56540bc550 0x7f56540be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f56540b2910 msgr2=0x7f56540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f56540b2910 0x7f56540b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5660009990 tx=0x7f56600096d0 comp rx=0 tx=0).stop 2024-05-11T12:22:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c103830 msgr2=0x7f566c19f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 --2- 172.21.15.47:0/3069219731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f566c103830 0x7f566c19f880 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b530 tx=0x7f565c00b840 comp rx=0 tx=0).stop 2024-05-11T12:22:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5670c14700 1 -- 172.21.15.47:0/3069219731 reap_dead start 2024-05-11T12:22:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 shutdown_connections 2024-05-11T12:22:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 >> 172.21.15.47:0/3069219731 conn(0x7f566c07a6c0 msgr2=0x7f566c109a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 shutdown_connections 2024-05-11T12:22:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:28.925+0000 7f5672677700 1 -- 172.21.15.47:0/3069219731 wait complete. 2024-05-11T12:22:28.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:28 smithi050 ceph-mon[34072]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:29.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:28 smithi064 ceph-mon[34537]: pgmap v334: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:30.075 INFO:teuthology.orchestra.run.smithi047.stdout:137438953541 2024-05-11T12:22:30.076 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-11T12:22:30.739 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:30 smithi047 ceph-mon[30375]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:30 smithi050 ceph-mon[34072]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:30 smithi064 ceph-mon[34537]: pgmap v335: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-05-11T12:22:31.579 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:31.580 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:32.093 INFO:teuthology.orchestra.run.smithi047.stdout:111669149770 2024-05-11T12:22:32.093 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-11T12:22:32.492 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:32.522 INFO:teuthology.orchestra.run.smithi047.stdout:231928234031 2024-05-11T12:22:32.522 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-11T12:22:32.744 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:32 smithi047 ceph-mon[30375]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:32.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:32 smithi050 ceph-mon[34072]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:33.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 -- 172.21.15.47:0/48214107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 msgr2=0x7fe0a4102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 --2- 172.21.15.47:0/48214107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 0x7fe0a4102900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe098009990 tx=0x7fe098009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 -- 172.21.15.47:0/48214107 shutdown_connections 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 --2- 172.21.15.47:0/48214107 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0a4102e40 0x7fe0a410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 --2- 172.21.15.47:0/48214107 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a41084c0 0x7fe0a41088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 --2- 172.21.15.47:0/48214107 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 0x7fe0a4102900 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 -- 172.21.15.47:0/48214107 >> 172.21.15.47:0/48214107 conn(0x7fe0a4079400 msgr2=0x7fe0a4079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 -- 172.21.15.47:0/48214107 shutdown_connections 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.032+0000 7fe0ac48f700 1 -- 172.21.15.47:0/48214107 wait complete. 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 Processor -- start 2024-05-11T12:22:33.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 -- start start 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 0x7fe0a419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0a4102e40 0x7fe0a419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a41084c0 0x7fe0a4199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a41125f0 con 0x7fe0a4102480 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4112470 con 0x7fe0a4102e40 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0ac48f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe0a4112770 con 0x7fe0a41084c0 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0aa22b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 0x7fe0a419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0aa22b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 0x7fe0a419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34568/0 (socket says 172.21.15.47:34568) 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0aa22b700 1 -- 172.21.15.47:0/564814746 learned_addr learned my addr 172.21.15.47:0/564814746 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0aaa2c700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a41084c0 0x7fe0a4199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.033+0000 7fe0a9a2a700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0a4102e40 0x7fe0a419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0aa22b700 1 -- 172.21.15.47:0/564814746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a41084c0 msgr2=0x7fe0a4199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0aa22b700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe0a41084c0 0x7fe0a4199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0aa22b700 1 -- 172.21.15.47:0/564814746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0a4102e40 msgr2=0x7fe0a419fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0aa22b700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe0a4102e40 0x7fe0a419fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0aa22b700 1 -- 172.21.15.47:0/564814746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0a419a2e0 con 0x7fe0a4102480 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0aa22b700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 0x7fe0a419f920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c970 tx=0x7fe0a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00078d0 con 0x7fe0a4102480 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0a0018480 con 0x7fe0a4102480 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0a419a5d0 con 0x7fe0a4102480 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a000f6b0 con 0x7fe0a4102480 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.034+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0a41a67f0 con 0x7fe0a4102480 2024-05-11T12:22:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.035+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0a4063c60 con 0x7fe0a4102480 2024-05-11T12:22:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.036+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe0a0018930 con 0x7fe0a4102480 2024-05-11T12:22:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.037+0000 7fe0977fe700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe0900b2910 0x7fe0900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.037+0000 7fe0a9a2a700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe0900b2910 0x7fe0900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.037+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fe0a00e2870 con 0x7fe0a4102480 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.037+0000 7fe0977fe700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] conn(0x7fe0900bc550 0x7fe0900be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.037+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 --> [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe0900bf0c0 con 0x7fe0900bc550 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.037+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe0a00e2cc0 con 0x7fe0a4102480 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.038+0000 7fe0aaa2c700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] conn(0x7fe0900bc550 0x7fe0900be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.038+0000 7fe0a9a2a700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe0900b2910 0x7fe0900b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b5c0 tx=0x7fe098005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.038+0000 7fe0aaa2c700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] conn(0x7fe0900bc550 0x7fe0900be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.039+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== osd.0 v2:172.21.15.47:6802/105259815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe0900bf0c0 con 0x7fe0900bc550 2024-05-11T12:22:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.057+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 --> [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0a40640e0 con 0x7fe0900bc550 2024-05-11T12:22:33.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0977fe700 1 -- 172.21.15.47:0/564814746 <== osd.0 v2:172.21.15.47:6802/105259815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe0a40640e0 con 0x7fe0900bc550 2024-05-11T12:22:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] conn(0x7fe0900bc550 msgr2=0x7fe0900be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] conn(0x7fe0900bc550 0x7fe0900be9f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe0900b2910 msgr2=0x7fe0900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe0900b2910 0x7fe0900b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe09800b5c0 tx=0x7fe098005960 comp rx=0 tx=0).stop 2024-05-11T12:22:33.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 msgr2=0x7fe0a419f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 --2- 172.21.15.47:0/564814746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0a4102480 0x7fe0a419f920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe0a000c970 tx=0x7fe0a000cc80 comp rx=0 tx=0).stop 2024-05-11T12:22:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0aaa2c700 1 -- 172.21.15.47:0/564814746 reap_dead start 2024-05-11T12:22:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 shutdown_connections 2024-05-11T12:22:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 >> 172.21.15.47:0/564814746 conn(0x7fe0a4079400 msgr2=0x7fe0a406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 shutdown_connections 2024-05-11T12:22:33.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.058+0000 7fe0ac48f700 1 -- 172.21.15.47:0/564814746 wait complete. 2024-05-11T12:22:33.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:32 smithi064 ceph-mon[34537]: pgmap v336: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:33.575 INFO:teuthology.orchestra.run.smithi047.stdout:184683593786 2024-05-11T12:22:33.575 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 -- 172.21.15.47:0/1185389728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 msgr2=0x7f38fc110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 --2- 172.21.15.47:0/1185389728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 0x7f38fc110480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f38f40099c0 tx=0x7f38f4009cd0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 -- 172.21.15.47:0/1185389728 shutdown_connections 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 --2- 172.21.15.47:0/1185389728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 0x7f38fc110480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 --2- 172.21.15.47:0/1185389728 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38fc103800 0x7f38fc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 --2- 172.21.15.47:0/1185389728 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38fc1041b0 0x7f38fc1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 -- 172.21.15.47:0/1185389728 >> 172.21.15.47:0/1185389728 conn(0x7f38fc07a6b0 msgr2=0x7f38fc07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:33.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 -- 172.21.15.47:0/1185389728 shutdown_connections 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 -- 172.21.15.47:0/1185389728 wait complete. 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.774+0000 7f390160c700 1 Processor -- start 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f390160c700 1 -- start start 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f390160c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38fc103800 0x7f38fc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f390160c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38fc1041b0 0x7f38fc19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f390160c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 0x7f38fc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f390160c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc1136f0 con 0x7f38fc108900 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f390160c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc113570 con 0x7f38fc1041b0 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f390160c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc113870 con 0x7f38fc103800 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 0x7f38fc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 0x7f38fc199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34590/0 (socket says 172.21.15.47:34590) 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 -- 172.21.15.47:0/677853910 learned_addr learned my addr 172.21.15.47:0/677853910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38faffd700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38fc103800 0x7f38fc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fa7fc700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38fc1041b0 0x7f38fc19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 -- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38fc103800 msgr2=0x7f38fc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f38fc103800 0x7f38fc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 -- 172.21.15.47:0/677853910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38fc1041b0 msgr2=0x7f38fc19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38fc1041b0 0x7f38fc19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 -- 172.21.15.47:0/677853910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc19a0f0 con 0x7f38fc108900 2024-05-11T12:22:33.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fa7fc700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f38fc1041b0 0x7f38fc19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.775+0000 7f38fb7fe700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 0x7f38fc199990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005910 tx=0x7f38f4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.776+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4027070 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.776+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38f4003f20 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.776+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc19a380 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.776+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4022c50 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.776+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a66a0 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.777+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38fc063ce0 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.778+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f38f402a070 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.779+0000 7f38f3fff700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f38e80b2990 0x7f38e80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.779+0000 7f38faffd700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f38e80b2990 0x7f38e80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.779+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f38f40e3390 con 0x7f38fc108900 2024-05-11T12:22:33.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.779+0000 7f38f3fff700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] conn(0x7f38e80bc5d0 0x7f38e80bea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.779+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 --> [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38f4007e80 con 0x7f38e80bc5d0 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.779+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f38f40e37e0 con 0x7f38fc108900 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.779+0000 7f38faffd700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f38e80b2990 0x7f38e80b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38fc19aad0 tx=0x7f38ec009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.780+0000 7f38fa7fc700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] conn(0x7f38e80bc5d0 0x7f38e80bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.780+0000 7f38fa7fc700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] conn(0x7f38e80bc5d0 0x7f38e80bea70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.781+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== osd.2 v2:172.21.15.64:6800/765449938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38f4007e80 con 0x7f38e80bc5d0 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.797+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 --> [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f38fc0640e0 con 0x7f38e80bc5d0 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f38f3fff700 1 -- 172.21.15.47:0/677853910 <== osd.2 v2:172.21.15.64:6800/765449938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f38fc0640e0 con 0x7f38e80bc5d0 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] conn(0x7f38e80bc5d0 msgr2=0x7f38e80bea70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] conn(0x7f38e80bc5d0 0x7f38e80bea70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:33.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f38e80b2990 msgr2=0x7f38e80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f38e80b2990 0x7f38e80b4e50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f38fc19aad0 tx=0x7f38ec009180 comp rx=0 tx=0).stop 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 msgr2=0x7f38fc199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 --2- 172.21.15.47:0/677853910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38fc108900 0x7f38fc199990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f38f4005910 tx=0x7f38f4009580 comp rx=0 tx=0).stop 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f38fb7fe700 1 -- 172.21.15.47:0/677853910 reap_dead start 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 shutdown_connections 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.798+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 >> 172.21.15.47:0/677853910 conn(0x7f38fc07a6b0 msgr2=0x7f38fc101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.799+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 shutdown_connections 2024-05-11T12:22:33.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:33.799+0000 7f390160c700 1 -- 172.21.15.47:0/677853910 wait complete. 2024-05-11T12:22:34.064 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:34.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:34 smithi047 ceph-mon[30375]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:34.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:34 smithi050 ceph-mon[34072]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:35.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:34 smithi064 ceph-mon[34537]: pgmap v337: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:36.735 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:36 smithi047 ceph-mon[30375]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:36.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:36 smithi050 ceph-mon[34072]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:37.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:36 smithi064 ceph-mon[34537]: pgmap v338: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:37.508 INFO:teuthology.orchestra.run.smithi047.stdout:34359738457 2024-05-11T12:22:37.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-11T12:22:38.885 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:38 smithi047 ceph-mon[30375]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:38.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:38 smithi050 ceph-mon[34072]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:39.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:38 smithi064 ceph-mon[34537]: pgmap v339: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:39.458 INFO:teuthology.orchestra.run.smithi047.stdout:77309411408 2024-05-11T12:22:39.458 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-11T12:22:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 -- 172.21.15.47:0/3017200051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 msgr2=0x7f552c103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 --2- 172.21.15.47:0/3017200051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c103a50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f551c009990 tx=0x7f551c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 -- 172.21.15.47:0/3017200051 shutdown_connections 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 --2- 172.21.15.47:0/3017200051 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f552c103f90 0x7f552c110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 --2- 172.21.15.47:0/3017200051 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f552c109650 0x7f552c109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 --2- 172.21.15.47:0/3017200051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c103a50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 -- 172.21.15.47:0/3017200051 >> 172.21.15.47:0/3017200051 conn(0x7f552c07a700 msgr2=0x7f552c07ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 -- 172.21.15.47:0/3017200051 shutdown_connections 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.027+0000 7f5533ed7700 1 -- 172.21.15.47:0/3017200051 wait complete. 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 Processor -- start 2024-05-11T12:22:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 -- start start 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c19f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f552c103f90 0x7f552c19fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f552c109650 0x7f552c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1137c0 con 0x7f552c1035d0 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f552c113640 con 0x7f552c103f90 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5533ed7700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f552c113940 con 0x7f552c109650 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5531c73700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c19f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5531c73700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c19f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37150/0 (socket says 172.21.15.47:37150) 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5531c73700 1 -- 172.21.15.47:0/753910947 learned_addr learned my addr 172.21.15.47:0/753910947 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:40.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5531472700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f552c103f90 0x7f552c19fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.028+0000 7f5532474700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f552c109650 0x7f552c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5531c73700 1 -- 172.21.15.47:0/753910947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f552c109650 msgr2=0x7f552c199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5531c73700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f552c109650 0x7f552c199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5531c73700 1 -- 172.21.15.47:0/753910947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f552c103f90 msgr2=0x7f552c19fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5531c73700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f552c103f90 0x7f552c19fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5531c73700 1 -- 172.21.15.47:0/753910947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c19a200 con 0x7f552c1035d0 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5532474700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f552c109650 0x7f552c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5531c73700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c19f8e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f552800c970 tx=0x7f552800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5522ffd700 1 -- 172.21.15.47:0/753910947 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528004220 con 0x7f552c1035d0 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c19a4f0 con 0x7f552c1035d0 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5522ffd700 1 -- 172.21.15.47:0/753910947 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5528007d50 con 0x7f552c1035d0 2024-05-11T12:22:40.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5522ffd700 1 -- 172.21.15.47:0/753910947 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5528003cc0 con 0x7f552c1035d0 2024-05-11T12:22:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.029+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a67b0 con 0x7f552c1035d0 2024-05-11T12:22:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.030+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c06a240 con 0x7f552c1035d0 2024-05-11T12:22:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.031+0000 7f5522ffd700 1 -- 172.21.15.47:0/753910947 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f55280043c0 con 0x7f552c1035d0 2024-05-11T12:22:40.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.032+0000 7f5522ffd700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55180b2910 0x7f55180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:40.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.032+0000 7f5531472700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55180b2910 0x7f55180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:40.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.032+0000 7f5522ffd700 1 -- 172.21.15.47:0/753910947 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7f55280e2400 con 0x7f552c1035d0 2024-05-11T12:22:40.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.033+0000 7f5531472700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55180b2910 0x7f55180b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f551c006010 tx=0x7f551c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:40.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.035+0000 7f5522ffd700 1 -- 172.21.15.47:0/753910947 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5528014030 con 0x7f552c1035d0 2024-05-11T12:22:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.256+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f552c19afd0 con 0x7f552c1035d0 2024-05-11T12:22:40.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.256+0000 7f5522ffd700 1 -- 172.21.15.47:0/753910947 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55280a9fa0 con 0x7f552c1035d0 2024-05-11T12:22:40.257 INFO:teuthology.orchestra.run.smithi047.stdout:158913790017 2024-05-11T12:22:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55180b2910 msgr2=0x7f55180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55180b2910 0x7f55180b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f551c006010 tx=0x7f551c01a040 comp rx=0 tx=0).stop 2024-05-11T12:22:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 msgr2=0x7f552c19f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c19f8e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f552800c970 tx=0x7f552800cd30 comp rx=0 tx=0).stop 2024-05-11T12:22:40.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 shutdown_connections 2024-05-11T12:22:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f552c109650 0x7f552c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f552c1035d0 0x7f552c19f8e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55180b2910 0x7f55180b4dd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 --2- 172.21.15.47:0/753910947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f552c103f90 0x7f552c19fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.258+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 >> 172.21.15.47:0/753910947 conn(0x7f552c07a700 msgr2=0x7f552c1021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.259+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 shutdown_connections 2024-05-11T12:22:40.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:40.259+0000 7f5533ed7700 1 -- 172.21.15.47:0/753910947 wait complete. 2024-05-11T12:22:40.880 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:40 smithi047 ceph-mon[30375]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:40.880 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:40 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/753910947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-11T12:22:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:40 smithi050 ceph-mon[34072]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:40 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/753910947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-11T12:22:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:40 smithi064 ceph-mon[34537]: pgmap v340: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:40 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/753910947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-11T12:22:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 -- 172.21.15.47:0/149677745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4501021e0 msgr2=0x7ff4501025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 --2- 172.21.15.47:0/149677745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4501021e0 0x7ff4501025c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009990 tx=0x7ff44c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 -- 172.21.15.47:0/149677745 shutdown_connections 2024-05-11T12:22:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 --2- 172.21.15.47:0/149677745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff45010fd60 0x7ff450112200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 --2- 172.21.15.47:0/149677745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4501021e0 0x7ff4501025c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 --2- 172.21.15.47:0/149677745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff450102b90 0x7ff45010f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 -- 172.21.15.47:0/149677745 >> 172.21.15.47:0/149677745 conn(0x7ff4500776c0 msgr2=0x7ff450077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 -- 172.21.15.47:0/149677745 shutdown_connections 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.160+0000 7ff4572fd700 1 -- 172.21.15.47:0/149677745 wait complete. 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 Processor -- start 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 -- start start 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4501021e0 0x7ff45019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff450102b90 0x7ff45019c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff45010fd60 0x7ff4501a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff4501159c0 con 0x7ff450102b90 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff450115840 con 0x7ff45010fd60 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff4572fd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff450115b40 con 0x7ff4501021e0 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff454898700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff450102b90 0x7ff45019c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff454898700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff450102b90 0x7ff45019c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37188/0 (socket says 172.21.15.47:37188) 2024-05-11T12:22:42.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff454898700 1 -- 172.21.15.47:0/2534240745 learned_addr learned my addr 172.21.15.47:0/2534240745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.161+0000 7ff455099700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4501021e0 0x7ff45019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff45589a700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff45010fd60 0x7ff4501a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff454898700 1 -- 172.21.15.47:0/2534240745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4501021e0 msgr2=0x7ff45019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff454898700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4501021e0 0x7ff45019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff454898700 1 -- 172.21.15.47:0/2534240745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff45010fd60 msgr2=0x7ff4501a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff454898700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff45010fd60 0x7ff4501a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff454898700 1 -- 172.21.15.47:0/2534240745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4501a0ed0 con 0x7ff450102b90 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff45589a700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff45010fd60 0x7ff4501a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff454898700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff450102b90 0x7ff45019c450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b500 tx=0x7ff44000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff4467fc700 1 -- 172.21.15.47:0/2534240745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440015020 con 0x7ff450102b90 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff4467fc700 1 -- 172.21.15.47:0/2534240745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff440011ce0 con 0x7ff450102b90 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4501a11c0 con 0x7ff450102b90 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff4467fc700 1 -- 172.21.15.47:0/2534240745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440010d40 con 0x7ff450102b90 2024-05-11T12:22:42.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.162+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4501a8a40 con 0x7ff450102b90 2024-05-11T12:22:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.163+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45006a240 con 0x7ff450102b90 2024-05-11T12:22:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.164+0000 7ff4467fc700 1 -- 172.21.15.47:0/2534240745 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff440018020 con 0x7ff450102b90 2024-05-11T12:22:42.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.165+0000 7ff4467fc700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff43c0b2910 0x7ff43c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.165+0000 7ff455099700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff43c0b2910 0x7ff43c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.165+0000 7ff4467fc700 1 -- 172.21.15.47:0/2534240745 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff4400e1fb0 con 0x7ff450102b90 2024-05-11T12:22:42.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.166+0000 7ff455099700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff43c0b2910 0x7ff43c0b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098e0 tx=0x7ff44c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:42.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.169+0000 7ff4467fc700 1 -- 172.21.15.47:0/2534240745 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff44001f020 con 0x7ff450102b90 2024-05-11T12:22:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.393+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff45002e570 con 0x7ff450102b90 2024-05-11T12:22:42.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.393+0000 7ff4467fc700 1 -- 172.21.15.47:0/2534240745 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff4400a9c00 con 0x7ff450102b90 2024-05-11T12:22:42.394 INFO:teuthology.orchestra.run.smithi047.stdout:304942678050 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff43c0b2910 msgr2=0x7ff43c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff43c0b2910 0x7ff43c0b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff44c0098e0 tx=0x7ff44c005960 comp rx=0 tx=0).stop 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff450102b90 msgr2=0x7ff45019c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff450102b90 0x7ff45019c450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff44000b500 tx=0x7ff44000b810 comp rx=0 tx=0).stop 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 shutdown_connections 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff43c0b2910 0x7ff43c0b4dd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff45010fd60 0x7ff4501a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff4501021e0 0x7ff45019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 --2- 172.21.15.47:0/2534240745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff450102b90 0x7ff45019c450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 >> 172.21.15.47:0/2534240745 conn(0x7ff4500776c0 msgr2=0x7ff450110470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 shutdown_connections 2024-05-11T12:22:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.396+0000 7ff4572fd700 1 -- 172.21.15.47:0/2534240745 wait complete. 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]: dispatch 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2534240745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-11T12:22:42.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:42 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/178907118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-11T12:22:42.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 -- 172.21.15.47:0/3340150847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f406006b230 msgr2=0x7f406006b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 --2- 172.21.15.47:0/3340150847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f406006b230 0x7f406006b6b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4048009990 tx=0x7f4048009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 -- 172.21.15.47:0/3340150847 shutdown_connections 2024-05-11T12:22:42.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 --2- 172.21.15.47:0/3340150847 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f406006bbf0 0x7f40601103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 --2- 172.21.15.47:0/3340150847 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f40601095c0 0x7f40601099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 --2- 172.21.15.47:0/3340150847 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f406006b230 0x7f406006b6b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 -- 172.21.15.47:0/3340150847 >> 172.21.15.47:0/3340150847 conn(0x7f4060079400 msgr2=0x7f4060079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:42.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 -- 172.21.15.47:0/3340150847 shutdown_connections 2024-05-11T12:22:42.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.876+0000 7f4065df1700 1 -- 172.21.15.47:0/3340150847 wait complete. 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 Processor -- start 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 -- start start 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f406006b230 0x7f406019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f406006bbf0 0x7f406019c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40601095c0 0x7f40601a06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f40601136f0 con 0x7f40601095c0 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4060113570 con 0x7f406006b230 2024-05-11T12:22:42.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f4065df1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4060113870 con 0x7f406006bbf0 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f405ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40601095c0 0x7f40601a06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f405ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40601095c0 0x7f40601a06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37218/0 (socket says 172.21.15.47:37218) 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.877+0000 7f405ffff700 1 -- 172.21.15.47:0/3421732382 learned_addr learned my addr 172.21.15.47:0/3421732382 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f4056ffd700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f406006bbf0 0x7f406019c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405f7fe700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f406006b230 0x7f406019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405ffff700 1 -- 172.21.15.47:0/3421732382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f406006bbf0 msgr2=0x7f406019c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405ffff700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f406006bbf0 0x7f406019c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405ffff700 1 -- 172.21.15.47:0/3421732382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f406006b230 msgr2=0x7f406019be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405ffff700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f406006b230 0x7f406019be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405ffff700 1 -- 172.21.15.47:0/3421732382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40601a0e30 con 0x7f40601095c0 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f4056ffd700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f406006bbf0 0x7f406019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405ffff700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40601095c0 0x7f40601a06f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4058007d70 tx=0x7f405800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405d7fa700 1 -- 172.21.15.47:0/3421732382 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058016020 con 0x7f40601095c0 2024-05-11T12:22:42.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40601a1120 con 0x7f40601095c0 2024-05-11T12:22:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405d7fa700 1 -- 172.21.15.47:0/3421732382 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f405800ad40 con 0x7f40601095c0 2024-05-11T12:22:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f405d7fa700 1 -- 172.21.15.47:0/3421732382 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580115e0 con 0x7f40601095c0 2024-05-11T12:22:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.878+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40601a89a0 con 0x7f40601095c0 2024-05-11T12:22:42.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.879+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406006a240 con 0x7f40601095c0 2024-05-11T12:22:42.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.881+0000 7f405d7fa700 1 -- 172.21.15.47:0/3421732382 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4058019020 con 0x7f40601095c0 2024-05-11T12:22:42.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.882+0000 7f405d7fa700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f404c0b2910 0x7f404c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:42.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.882+0000 7f405f7fe700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f404c0b2910 0x7f404c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:42.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.882+0000 7f405d7fa700 1 -- 172.21.15.47:0/3421732382 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8443+0+0 (secure 0 0 0) 0x7f40580e2720 con 0x7f40601095c0 2024-05-11T12:22:42.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.882+0000 7f405f7fe700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f404c0b2910 0x7f404c0b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f405000bbb0 tx=0x7f405000b590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:42.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.886+0000 7f405d7fa700 1 -- 172.21.15.47:0/3421732382 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4058020020 con 0x7f40601095c0 2024-05-11T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-05-11T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-11T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-11T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-11T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]: dispatch 2024-05-11T12:22:42.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-11T12:22:42.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2534240745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-11T12:22:42.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:42 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/178907118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-11T12:22:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 -- 172.21.15.47:0/2402829070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070106f00 msgr2=0x7ff07010ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 --2- 172.21.15.47:0/2402829070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070106f00 0x7ff07010ead0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff06800b0d0 tx=0x7ff06800b3e0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 -- 172.21.15.47:0/2402829070 shutdown_connections 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 --2- 172.21.15.47:0/2402829070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070106f00 0x7ff07010ead0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 --2- 172.21.15.47:0/2402829070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff070101e70 0x7ff070102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 --2- 172.21.15.47:0/2402829070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff070102820 0x7ff070106820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 -- 172.21.15.47:0/2402829070 >> 172.21.15.47:0/2402829070 conn(0x7ff070078ac0 msgr2=0x7ff0700ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 -- 172.21.15.47:0/2402829070 shutdown_connections 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.363+0000 7ff076c7c700 1 -- 172.21.15.47:0/2402829070 wait complete. 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 Processor -- start 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 -- start start 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070101e70 0x7ff07019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff070102820 0x7ff07019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff070106f00 0x7ff070197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff070111560 con 0x7ff070101e70 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701113e0 con 0x7ff070106f00 2024-05-11T12:22:43.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff076c7c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0701116e0 con 0x7ff070102820 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff074a18700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070101e70 0x7ff07019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff074a18700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070101e70 0x7ff07019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37194/0 (socket says 172.21.15.47:37194) 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.364+0000 7ff074a18700 1 -- 172.21.15.47:0/178907118 learned_addr learned my addr 172.21.15.47:0/178907118 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff06ffff700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff070102820 0x7ff07019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff075219700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff070106f00 0x7ff070197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff074a18700 1 -- 172.21.15.47:0/178907118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff070102820 msgr2=0x7ff07019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff074a18700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff070102820 0x7ff07019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff074a18700 1 -- 172.21.15.47:0/178907118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff070106f00 msgr2=0x7ff070197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff074a18700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff070106f00 0x7ff070197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff074a18700 1 -- 172.21.15.47:0/178907118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff070197fe0 con 0x7ff070101e70 2024-05-11T12:22:43.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff074a18700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070101e70 0x7ff07019d6c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c920 tx=0x7ff06400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stdout:206158430264 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff06dffb700 1 -- 172.21.15.47:0/178907118 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064007b00 con 0x7ff070101e70 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff06dffb700 1 -- 172.21.15.47:0/178907118 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff064004ce0 con 0x7ff070101e70 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0701982d0 con 0x7ff070101e70 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff06dffb700 1 -- 172.21.15.47:0/178907118 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064005650 con 0x7ff070101e70 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.365+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0701a4590 con 0x7ff070101e70 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.366+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff070103300 con 0x7ff070101e70 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.369+0000 7ff06dffb700 1 -- 172.21.15.47:0/178907118 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff064004750 con 0x7ff070101e70 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.369+0000 7ff06dffb700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0600b2990 0x7ff0600b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:43.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.369+0000 7ff06dffb700 1 -- 172.21.15.47:0/178907118 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7ff0640e1f20 con 0x7ff070101e70 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.369+0000 7ff06ffff700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0600b2990 0x7ff0600b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.370+0000 7ff06ffff700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0600b2990 0x7ff0600b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff070108980 tx=0x7ff05c00c6a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.374+0000 7ff06dffb700 1 -- 172.21.15.47:0/178907118 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff064014030 con 0x7ff070101e70 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.605+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ff070198db0 con 0x7ff070101e70 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.605+0000 7ff06dffb700 1 -- 172.21.15.47:0/178907118 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff0640a9ac0 con 0x7ff070101e70 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0600b2990 msgr2=0x7ff0600b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0600b2990 0x7ff0600b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff070108980 tx=0x7ff05c00c6a0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070101e70 msgr2=0x7ff07019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070101e70 0x7ff07019d6c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff06400c920 tx=0x7ff06400cce0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 shutdown_connections 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0600b2990 0x7ff0600b4e50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff070106f00 0x7ff070197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff070101e70 0x7ff07019d6c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 --2- 172.21.15.47:0/178907118 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff070102820 0x7ff07019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 >> 172.21.15.47:0/178907118 conn(0x7ff070078ac0 msgr2=0x7ff070105630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:43.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 shutdown_connections 2024-05-11T12:22:43.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:42.608+0000 7ff076c7c700 1 -- 172.21.15.47:0/178907118 wait complete. 2024-05-11T12:22:43.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.106+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f406019d180 con 0x7f40601095c0 2024-05-11T12:22:43.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.106+0000 7f405d7fa700 1 -- 172.21.15.47:0/3421732382 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f406019d180 con 0x7f40601095c0 2024-05-11T12:22:43.108 INFO:teuthology.orchestra.run.smithi047.stdout:257698037805 2024-05-11T12:22:43.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.108+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f404c0b2910 msgr2=0x7f404c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:43.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.108+0000 7f4065df1700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f404c0b2910 0x7f404c0b4dd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f405000bbb0 tx=0x7f405000b590 comp rx=0 tx=0).stop 2024-05-11T12:22:43.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.108+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40601095c0 msgr2=0x7f40601a06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:43.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.108+0000 7f4065df1700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40601095c0 0x7f40601a06f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4058007d70 tx=0x7f405800a340 comp rx=0 tx=0).stop 2024-05-11T12:22:43.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.108+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 shutdown_connections 2024-05-11T12:22:43.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.109+0000 7f4065df1700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40601095c0 0x7f40601a06f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.109+0000 7f4065df1700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f404c0b2910 0x7f404c0b4dd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.109+0000 7f4065df1700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f406006b230 0x7f406019be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.109+0000 7f4065df1700 1 --2- 172.21.15.47:0/3421732382 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f406006bbf0 0x7f406019c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:43.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.109+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 >> 172.21.15.47:0/3421732382 conn(0x7f4060079400 msgr2=0x7f406010e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:43.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.109+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 shutdown_connections 2024-05-11T12:22:43.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:43.109+0000 7f4065df1700 1 -- 172.21.15.47:0/3421732382 wait complete. 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: pgmap v341: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]: dispatch 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]: dispatch 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2534240745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-11T12:22:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:42 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/178907118' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-11T12:22:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-05-11T12:22:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-11T12:22:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-11T12:22:43.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-11T12:22:43.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]': finished 2024-05-11T12:22:43.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-11T12:22:43.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: osdmap e82: 12 total, 12 up, 12 in 2024-05-11T12:22:43.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:43 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3421732382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-11T12:22:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-05-11T12:22:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-11T12:22:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-11T12:22:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-11T12:22:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]': finished 2024-05-11T12:22:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-11T12:22:44.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: osdmap e82: 12 total, 12 up, 12 in 2024-05-11T12:22:44.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:43 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3421732382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-11T12:22:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-05-11T12:22:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-05-11T12:22:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 7]}]': finished 2024-05-11T12:22:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-11T12:22:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [1, 9]}]': finished 2024-05-11T12:22:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-05-11T12:22:44.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: osdmap e82: 12 total, 12 up, 12 in 2024-05-11T12:22:44.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:43 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3421732382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-11T12:22:44.209 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037799 got 257698037805 for osd.9 2024-05-11T12:22:44.210 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:44.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:44 smithi047 ceph-mon[30375]: pgmap v343: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:22:44.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:44 smithi047 ceph-mon[30375]: osdmap e83: 12 total, 12 up, 12 in 2024-05-11T12:22:44.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790012 got 158913790017 for osd.5 2024-05-11T12:22:44.933 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:44.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:44 smithi050 ceph-mon[34072]: pgmap v343: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:22:44.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:44 smithi050 ceph-mon[34072]: osdmap e83: 12 total, 12 up, 12 in 2024-05-11T12:22:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 -- 172.21.15.47:0/592882201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 msgr2=0x7fd588110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 --2- 172.21.15.47:0/592882201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 0x7fd588110480 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd58000b0d0 tx=0x7fd58000b3e0 comp rx=0 tx=0).stop 2024-05-11T12:22:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 -- 172.21.15.47:0/592882201 shutdown_connections 2024-05-11T12:22:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 --2- 172.21.15.47:0/592882201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 0x7fd588110480 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:45.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 --2- 172.21.15.47:0/592882201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd588103800 0x7fd588103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 --2- 172.21.15.47:0/592882201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd5881041b0 0x7fd5881081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 -- 172.21.15.47:0/592882201 >> 172.21.15.47:0/592882201 conn(0x7fd58807a6b0 msgr2=0x7fd58807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 -- 172.21.15.47:0/592882201 shutdown_connections 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 -- 172.21.15.47:0/592882201 wait complete. 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.111+0000 7fd58d478700 1 Processor -- start 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd58d478700 1 -- start start 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd58d478700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd588103800 0x7fd58819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd58d478700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5881041b0 0x7fd58819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd58d478700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 0x7fd588199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd58d478700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881136f0 con 0x7fd588108900 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd58d478700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd588113570 con 0x7fd588103800 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd58d478700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd588113870 con 0x7fd5881041b0 2024-05-11T12:22:45.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 0x7fd588199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 0x7fd588199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37168/0 (socket says 172.21.15.47:37168) 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 -- 172.21.15.47:0/2049241068 learned_addr learned my addr 172.21.15.47:0/2049241068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd586ffd700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd588103800 0x7fd58819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5867fc700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5881041b0 0x7fd58819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 -- 172.21.15.47:0/2049241068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5881041b0 msgr2=0x7fd58819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd5881041b0 0x7fd58819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 -- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd588103800 msgr2=0x7fd58819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd588103800 0x7fd58819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.112+0000 7fd5877fe700 1 -- 172.21.15.47:0/2049241068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58819a090 con 0x7fd588108900 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.113+0000 7fd5877fe700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 0x7fd588199930 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd580009850 tx=0x7fd580007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.113+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58001f070 con 0x7fd588108900 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.113+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd580004c50 con 0x7fd588108900 2024-05-11T12:22:45.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.113+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58819a320 con 0x7fd588108900 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.113+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58001b690 con 0x7fd588108900 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.113+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a6640 con 0x7fd588108900 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.114+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd588063ce0 con 0x7fd588108900 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.115+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd580022070 con 0x7fd588108900 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.116+0000 7fd56ffff700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd5700b2910 0x7fd5700b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.116+0000 7fd586ffd700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd5700b2910 0x7fd5700b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.116+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8269+0+0 (secure 0 0 0) 0x7fd58000f020 con 0x7fd588108900 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.116+0000 7fd56ffff700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] conn(0x7fd5700bc550 0x7fd5700be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.116+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 --> [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5700bf0c0 con 0x7fd5700bc550 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.116+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fd5800e3b80 con 0x7fd588108900 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.116+0000 7fd586ffd700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd5700b2910 0x7fd5700b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd58819aa70 tx=0x7fd578009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:45.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.117+0000 7fd5867fc700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] conn(0x7fd5700bc550 0x7fd5700be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.117+0000 7fd5867fc700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] conn(0x7fd5700bc550 0x7fd5700be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.118+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== osd.1 v2:172.21.15.50:6800/4057338653 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5700bf0c0 con 0x7fd5700bc550 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.134+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 --> [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd588064160 con 0x7fd5700bc550 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd56ffff700 1 -- 172.21.15.47:0/2049241068 <== osd.1 v2:172.21.15.50:6800/4057338653 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd588064160 con 0x7fd5700bc550 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] conn(0x7fd5700bc550 msgr2=0x7fd5700be9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] conn(0x7fd5700bc550 0x7fd5700be9f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd5700b2910 msgr2=0x7fd5700b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd5700b2910 0x7fd5700b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd58819aa70 tx=0x7fd578009260 comp rx=0 tx=0).stop 2024-05-11T12:22:45.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 msgr2=0x7fd588199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 --2- 172.21.15.47:0/2049241068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd588108900 0x7fd588199930 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd580009850 tx=0x7fd580007a20 comp rx=0 tx=0).stop 2024-05-11T12:22:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd5877fe700 1 -- 172.21.15.47:0/2049241068 reap_dead start 2024-05-11T12:22:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 shutdown_connections 2024-05-11T12:22:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.135+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 >> 172.21.15.47:0/2049241068 conn(0x7fd58807a6b0 msgr2=0x7fd5881018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.136+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 shutdown_connections 2024-05-11T12:22:45.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:41.136+0000 7fd58d478700 1 -- 172.21.15.47:0/2049241068 wait complete. 2024-05-11T12:22:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:44 smithi064 ceph-mon[34537]: pgmap v343: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:22:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:44 smithi064 ceph-mon[34537]: osdmap e83: 12 total, 12 up, 12 in 2024-05-11T12:22:45.753 INFO:teuthology.orchestra.run.smithi047.stdout:55834574934 2024-05-11T12:22:45.753 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-11T12:22:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.966+0000 7fb30ebe0700 1 -- 172.21.15.47:0/2980294652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3081095c0 msgr2=0x7fb3081099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.966+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/2980294652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3081095c0 0x7fb3081099a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009990 tx=0x7fb2fc009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.967+0000 7fb30ebe0700 1 -- 172.21.15.47:0/2980294652 shutdown_connections 2024-05-11T12:22:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.967+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/2980294652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb30806bbf0 0x7fb3081103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.967+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/2980294652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3081095c0 0x7fb3081099a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.967+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/2980294652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb30806b230 0x7fb30806b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.967+0000 7fb30ebe0700 1 -- 172.21.15.47:0/2980294652 >> 172.21.15.47:0/2980294652 conn(0x7fb308079400 msgr2=0x7fb308079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.967+0000 7fb30ebe0700 1 -- 172.21.15.47:0/2980294652 shutdown_connections 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.967+0000 7fb30ebe0700 1 -- 172.21.15.47:0/2980294652 wait complete. 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 Processor -- start 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 -- start start 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb30806b230 0x7fb30819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb30806bbf0 0x7fb30819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3081095c0 0x7fb308199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081136f0 con 0x7fb30806b230 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb308113570 con 0x7fb30806bbf0 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30ebe0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb308113870 con 0x7fb3081095c0 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30c97c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb30806b230 0x7fb30819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:46.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30c97c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb30806b230 0x7fb30819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37238/0 (socket says 172.21.15.47:37238) 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.968+0000 7fb30c97c700 1 -- 172.21.15.47:0/3159715478 learned_addr learned my addr 172.21.15.47:0/3159715478 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30d17d700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3081095c0 0x7fb308199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb307fff700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb30806bbf0 0x7fb30819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30c97c700 1 -- 172.21.15.47:0/3159715478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3081095c0 msgr2=0x7fb308199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30c97c700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3081095c0 0x7fb308199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30c97c700 1 -- 172.21.15.47:0/3159715478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb30806bbf0 msgr2=0x7fb30819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30c97c700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb30806bbf0 0x7fb30819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30c97c700 1 -- 172.21.15.47:0/3159715478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30819a1b0 con 0x7fb30806b230 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb307fff700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb30806bbf0 0x7fb30819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30c97c700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb30806b230 0x7fb30819f890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc000c00 tx=0x7fb2fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb305ffb700 1 -- 172.21.15.47:0/3159715478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc027070 con 0x7fb30806b230 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb305ffb700 1 -- 172.21.15.47:0/3159715478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2fc003fa0 con 0x7fb30806b230 2024-05-11T12:22:46.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30819a440 con 0x7fb30806b230 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb305ffb700 1 -- 172.21.15.47:0/3159715478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc022db0 con 0x7fb30806b230 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.969+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3081a6760 con 0x7fb30806b230 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.970+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30810baa0 con 0x7fb30806b230 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.972+0000 7fb305ffb700 1 -- 172.21.15.47:0/3159715478 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb2fc02a070 con 0x7fb30806b230 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.973+0000 7fb305ffb700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb2f80b2910 0x7fb2f80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.973+0000 7fb307fff700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb2f80b2910 0x7fb2f80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stdout:279172874280 2024-05-11T12:22:46.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.973+0000 7fb305ffb700 1 -- 172.21.15.47:0/3159715478 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb2fc0e3590 con 0x7fb30806b230 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.973+0000 7fb307fff700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb2f80b2910 0x7fb2f80b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005fd0 tx=0x7fb2f4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:45.980+0000 7fb305ffb700 1 -- 172.21.15.47:0/3159715478 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2fc013070 con 0x7fb30806b230 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.201+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb30819af80 con 0x7fb30806b230 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.201+0000 7fb305ffb700 1 -- 172.21.15.47:0/3159715478 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb2fc0ab0b0 con 0x7fb30806b230 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb2f80b2910 msgr2=0x7fb2f80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb2f80b2910 0x7fb2f80b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4005fd0 tx=0x7fb2f4005ee0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb30806b230 msgr2=0x7fb30819f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb30806b230 0x7fb30819f890 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc000c00 tx=0x7fb2fc0047b0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 shutdown_connections 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb3081095c0 0x7fb308199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb30806b230 0x7fb30819f890 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb2f80b2910 0x7fb2f80b4dd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 --2- 172.21.15.47:0/3159715478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb30806bbf0 0x7fb30819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:46.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 >> 172.21.15.47:0/3159715478 conn(0x7fb308079400 msgr2=0x7fb308101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:46.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 shutdown_connections 2024-05-11T12:22:46.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:46.204+0000 7fb30ebe0700 1 -- 172.21.15.47:0/3159715478 wait complete. 2024-05-11T12:22:46.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:46 smithi047 ceph-mon[30375]: pgmap v345: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:46.893 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:46 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3159715478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-11T12:22:46.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874274 got 279172874280 for osd.10 2024-05-11T12:22:46.920 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:46.944 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430259 got 206158430264 for osd.7 2024-05-11T12:22:46.944 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:46 smithi050 ceph-mon[34072]: pgmap v345: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:46 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3159715478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-11T12:22:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:46.966 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:47.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:46 smithi064 ceph-mon[34537]: pgmap v345: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:47.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:46 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3159715478' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-11T12:22:47.405 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678046 got 304942678050 for osd.11 2024-05-11T12:22:47.405 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:47.413 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:47.445 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:48.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:48 smithi050 ceph-mon[34072]: pgmap v346: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:49.019 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:48 smithi047 ceph-mon[30375]: pgmap v346: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:49.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:48 smithi064 ceph-mon[34537]: pgmap v346: 97 pgs: 1 activating, 96 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:22:50.909 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:50 smithi047 ceph-mon[30375]: pgmap v347: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-05-11T12:22:50.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:50 smithi050 ceph-mon[34072]: pgmap v347: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-05-11T12:22:51.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:50 smithi064 ceph-mon[34537]: pgmap v347: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-05-11T12:22:51.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 -- 172.21.15.47:0/1164660505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888108920 msgr2=0x7ff8881104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 --2- 172.21.15.47:0/1164660505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888108920 0x7ff8881104a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff88000b0d0 tx=0x7ff88000b3e0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 -- 172.21.15.47:0/1164660505 shutdown_connections 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 --2- 172.21.15.47:0/1164660505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888108920 0x7ff8881104a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 --2- 172.21.15.47:0/1164660505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff888103830 0x7ff888103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 --2- 172.21.15.47:0/1164660505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8881041e0 0x7ff8881081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 -- 172.21.15.47:0/1164660505 >> 172.21.15.47:0/1164660505 conn(0x7ff88807a6c0 msgr2=0x7ff88807aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 -- 172.21.15.47:0/1164660505 shutdown_connections 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.330+0000 7ff890673700 1 -- 172.21.15.47:0/1164660505 wait complete. 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 Processor -- start 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 -- start start 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888103830 0x7ff88819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8881041e0 0x7ff88819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:51.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff888108920 0x7ff888199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff888113770 con 0x7ff888103830 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881135f0 con 0x7ff8881041e0 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff890673700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881138f0 con 0x7ff888108920 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888103830 0x7ff88819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888103830 0x7ff88819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37008/0 (socket says 172.21.15.47:37008) 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 -- 172.21.15.47:0/3376492594 learned_addr learned my addr 172.21.15.47:0/3376492594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88ec10700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff888108920 0x7ff888199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88dc0e700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8881041e0 0x7ff88819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 -- 172.21.15.47:0/3376492594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff888108920 msgr2=0x7ff888199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff888108920 0x7ff888199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 -- 172.21.15.47:0/3376492594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8881041e0 msgr2=0x7ff88819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8881041e0 0x7ff88819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88e40f700 1 -- 172.21.15.47:0/3376492594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88819a090 con 0x7ff888103830 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.331+0000 7ff88dc0e700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8881041e0 0x7ff88819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:51.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.332+0000 7ff88e40f700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888103830 0x7ff88819f770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff88400b530 tx=0x7ff88400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.332+0000 7ff87f7fe700 1 -- 172.21.15.47:0/3376492594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884015020 con 0x7ff888103830 2024-05-11T12:22:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.332+0000 7ff87f7fe700 1 -- 172.21.15.47:0/3376492594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff884011ce0 con 0x7ff888103830 2024-05-11T12:22:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.332+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88819a380 con 0x7ff888103830 2024-05-11T12:22:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.332+0000 7ff87f7fe700 1 -- 172.21.15.47:0/3376492594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884010d20 con 0x7ff888103830 2024-05-11T12:22:51.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.332+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881a6640 con 0x7ff888103830 2024-05-11T12:22:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.333+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88806a240 con 0x7ff888103830 2024-05-11T12:22:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.334+0000 7ff87f7fe700 1 -- 172.21.15.47:0/3376492594 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff884018020 con 0x7ff888103830 2024-05-11T12:22:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.335+0000 7ff87f7fe700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8740b2910 0x7ff8740b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.335+0000 7ff88dc0e700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8740b2910 0x7ff8740b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.335+0000 7ff87f7fe700 1 -- 172.21.15.47:0/3376492594 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff8840e1c30 con 0x7ff888103830 2024-05-11T12:22:51.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.335+0000 7ff88dc0e700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8740b2910 0x7ff8740b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff878005fd0 tx=0x7ff878005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:51.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.339+0000 7ff87f7fe700 1 -- 172.21.15.47:0/3376492594 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff88401f020 con 0x7ff888103830 2024-05-11T12:22:51.562 INFO:teuthology.orchestra.run.smithi047.stdout:231928234035 2024-05-11T12:22:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.562+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff88819ae60 con 0x7ff888103830 2024-05-11T12:22:51.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.562+0000 7ff87f7fe700 1 -- 172.21.15.47:0/3376492594 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff8840a97d0 con 0x7ff888103830 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8740b2910 msgr2=0x7ff8740b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8740b2910 0x7ff8740b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff878005fd0 tx=0x7ff878005e20 comp rx=0 tx=0).stop 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888103830 msgr2=0x7ff88819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888103830 0x7ff88819f770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff88400b530 tx=0x7ff88400b8f0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 shutdown_connections 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff888108920 0x7ff888199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff888103830 0x7ff88819f770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8740b2910 0x7ff8740b4dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 --2- 172.21.15.47:0/3376492594 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8881041e0 0x7ff88819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 >> 172.21.15.47:0/3376492594 conn(0x7ff88807a6c0 msgr2=0x7ff8881014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 shutdown_connections 2024-05-11T12:22:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:51.565+0000 7ff890673700 1 -- 172.21.15.47:0/3376492594 wait complete. 2024-05-11T12:22:51.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:51 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3376492594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-11T12:22:51.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:51 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3376492594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-11T12:22:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:51 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3376492594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-11T12:22:52.936 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:52 smithi047 ceph-mon[30375]: pgmap v348: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-11T12:22:52.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:52 smithi050 ceph-mon[34072]: pgmap v348: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-11T12:22:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:52 smithi064 ceph-mon[34537]: pgmap v348: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 -- 172.21.15.47:0/1619045468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 msgr2=0x7fcecc06c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 --2- 172.21.15.47:0/1619045468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc06c1a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0099c0 tx=0x7fcebc009cd0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 -- 172.21.15.47:0/1619045468 shutdown_connections 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 --2- 172.21.15.47:0/1619045468 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc06c6e0 0x7fcecc110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 --2- 172.21.15.47:0/1619045468 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcecc109640 0x7fcecc109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 --2- 172.21.15.47:0/1619045468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc06c1a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 -- 172.21.15.47:0/1619045468 >> 172.21.15.47:0/1619045468 conn(0x7fcecc0794f0 msgr2=0x7fcecc079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 -- 172.21.15.47:0/1619045468 shutdown_connections 2024-05-11T12:22:53.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 -- 172.21.15.47:0/1619045468 wait complete. 2024-05-11T12:22:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.198+0000 7fced444d700 1 Processor -- start 2024-05-11T12:22:53.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced444d700 1 -- start start 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced444d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced444d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcecc06c6e0 0x7fcecc19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced444d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc109640 0x7fcecc1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced444d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1137a0 con 0x7fcecc06bd20 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced444d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc113620 con 0x7fcecc06c6e0 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced444d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc113920 con 0x7fcecc109640 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc19f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37040/0 (socket says 172.21.15.47:37040) 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 -- 172.21.15.47:0/1971548581 learned_addr learned my addr 172.21.15.47:0/1971548581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced29ea700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc109640 0x7fcecc1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced19e8700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcecc06c6e0 0x7fcecc19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 -- 172.21.15.47:0/1971548581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc109640 msgr2=0x7fcecc1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc109640 0x7fcecc1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 -- 172.21.15.47:0/1971548581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcecc06c6e0 msgr2=0x7fcecc19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcecc06c6e0 0x7fcecc19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 -- 172.21.15.47:0/1971548581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc19a100 con 0x7fcecc06bd20 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.199+0000 7fced21e9700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc19f7e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcec800d980 tx=0x7fcec800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.200+0000 7fcec37fe700 1 -- 172.21.15.47:0/1971548581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8017070 con 0x7fcecc06bd20 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.200+0000 7fcec37fe700 1 -- 172.21.15.47:0/1971548581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcec8005020 con 0x7fcecc06bd20 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.200+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc19a3f0 con 0x7fcecc06bd20 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.200+0000 7fcec37fe700 1 -- 172.21.15.47:0/1971548581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8009b50 con 0x7fcecc06bd20 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.200+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc1a66b0 con 0x7fcecc06bd20 2024-05-11T12:22:53.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.201+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcecc06a240 con 0x7fcecc06bd20 2024-05-11T12:22:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.202+0000 7fcec37fe700 1 -- 172.21.15.47:0/1971548581 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcec80051d0 con 0x7fcecc06bd20 2024-05-11T12:22:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.203+0000 7fcec37fe700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fceb80b2990 0x7fceb80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.203+0000 7fced19e8700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fceb80b2990 0x7fceb80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.203+0000 7fcec37fe700 1 -- 172.21.15.47:0/1971548581 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fcec80e2750 con 0x7fcecc06bd20 2024-05-11T12:22:53.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.203+0000 7fced19e8700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fceb80b2990 0x7fceb80b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcecc079aa0 tx=0x7fcebc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:53.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.206+0000 7fcec37fe700 1 -- 172.21.15.47:0/1971548581 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcec8020020 con 0x7fcecc06bd20 2024-05-11T12:22:53.422 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:53.422 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:22:53.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.429+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcecc02e570 con 0x7fcecc06bd20 2024-05-11T12:22:53.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.429+0000 7fcec37fe700 1 -- 172.21.15.47:0/1971548581 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcec80054f0 con 0x7fcecc06bd20 2024-05-11T12:22:53.429 INFO:teuthology.orchestra.run.smithi047.stdout:184683593791 2024-05-11T12:22:53.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fceb80b2990 msgr2=0x7fceb80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:53.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fceb80b2990 0x7fceb80b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcecc079aa0 tx=0x7fcebc01a040 comp rx=0 tx=0).stop 2024-05-11T12:22:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 msgr2=0x7fcecc19f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc19f7e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcec800d980 tx=0x7fcec800dc90 comp rx=0 tx=0).stop 2024-05-11T12:22:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 shutdown_connections 2024-05-11T12:22:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcecc109640 0x7fcecc1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcecc06bd20 0x7fcecc19f7e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fceb80b2990 0x7fceb80b4e50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 --2- 172.21.15.47:0/1971548581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcecc06c6e0 0x7fcecc19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:53.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 >> 172.21.15.47:0/1971548581 conn(0x7fcecc0794f0 msgr2=0x7fcecc100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:53.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 shutdown_connections 2024-05-11T12:22:53.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:53.432+0000 7fced444d700 1 -- 172.21.15.47:0/1971548581 wait complete. 2024-05-11T12:22:53.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:53 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1971548581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-11T12:22:53.865 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234031 got 231928234035 for osd.8 2024-05-11T12:22:53.866 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:53.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:53 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1971548581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-11T12:22:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:53 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1971548581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-11T12:22:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.622+0000 7fc40a265700 1 -- 172.21.15.47:0/3875782047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406b230 msgr2=0x7fc40406b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.622+0000 7fc40a265700 1 --2- 172.21.15.47:0/3875782047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406b230 0x7fc40406b6b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009990 tx=0x7fc3ec009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.622+0000 7fc40a265700 1 -- 172.21.15.47:0/3875782047 shutdown_connections 2024-05-11T12:22:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.622+0000 7fc40a265700 1 --2- 172.21.15.47:0/3875782047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc40406bbf0 0x7fc4041103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.622+0000 7fc40a265700 1 --2- 172.21.15.47:0/3875782047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4041095c0 0x7fc4041099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.622+0000 7fc40a265700 1 --2- 172.21.15.47:0/3875782047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406b230 0x7fc40406b6b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.622+0000 7fc40a265700 1 -- 172.21.15.47:0/3875782047 >> 172.21.15.47:0/3875782047 conn(0x7fc404079400 msgr2=0x7fc404079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 -- 172.21.15.47:0/3875782047 shutdown_connections 2024-05-11T12:22:54.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 -- 172.21.15.47:0/3875782047 wait complete. 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 Processor -- start 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 -- start start 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc40406b230 0x7fc40419fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406bbf0 0x7fc40419ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4041095c0 0x7fc404199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc404113750 con 0x7fc40406bbf0 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041135d0 con 0x7fc40406b230 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc40a265700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc4041138d0 con 0x7fc4041095c0 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.623+0000 7fc3fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406bbf0 0x7fc40419ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406bbf0 0x7fc40419ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37058/0 (socket says 172.21.15.47:37058) 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 -- 172.21.15.47:0/2773422011 learned_addr learned my addr 172.21.15.47:0/2773422011 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:54.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc408802700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4041095c0 0x7fc404199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc403fff700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc40406b230 0x7fc40419fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 -- 172.21.15.47:0/2773422011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4041095c0 msgr2=0x7fc404199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4041095c0 0x7fc404199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 -- 172.21.15.47:0/2773422011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc40406b230 msgr2=0x7fc40419fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc40406b230 0x7fc40419fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 -- 172.21.15.47:0/2773422011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40419a3f0 con 0x7fc40406bbf0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc403fff700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc40406b230 0x7fc40419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc408802700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4041095c0 0x7fc404199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc3fbfff700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406bbf0 0x7fc40419ff80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc401ffb700 1 -- 172.21.15.47:0/2773422011 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec027070 con 0x7fc40406bbf0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc401ffb700 1 -- 172.21.15.47:0/2773422011 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3ec0041d0 con 0x7fc40406bbf0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40419a680 con 0x7fc40406bbf0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc401ffb700 1 -- 172.21.15.47:0/2773422011 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00f460 con 0x7fc40406bbf0 2024-05-11T12:22:54.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.624+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4041a69c0 con 0x7fc40406bbf0 2024-05-11T12:22:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.625+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40419a9e0 con 0x7fc40406bbf0 2024-05-11T12:22:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.627+0000 7fc401ffb700 1 -- 172.21.15.47:0/2773422011 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc3ec02a070 con 0x7fc40406bbf0 2024-05-11T12:22:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.627+0000 7fc401ffb700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3f00b2990 0x7fc3f00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:54.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.627+0000 7fc403fff700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3f00b2990 0x7fc3f00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:54.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.627+0000 7fc401ffb700 1 -- 172.21.15.47:0/2773422011 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc3ec0e3830 con 0x7fc40406bbf0 2024-05-11T12:22:54.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.628+0000 7fc403fff700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3f00b2990 0x7fc3f00b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4007900 tx=0x7fc3f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:54.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.631+0000 7fc401ffb700 1 -- 172.21.15.47:0/2773422011 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc3ec031070 con 0x7fc40406bbf0 2024-05-11T12:22:54.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.851+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fc40419afd0 con 0x7fc40406bbf0 2024-05-11T12:22:54.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.851+0000 7fc401ffb700 1 -- 172.21.15.47:0/2773422011 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc3ec013070 con 0x7fc40406bbf0 2024-05-11T12:22:54.851 INFO:teuthology.orchestra.run.smithi047.stdout:137438953546 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3f00b2990 msgr2=0x7fc3f00b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3f00b2990 0x7fc3f00b4e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4007900 tx=0x7fc3f4008040 comp rx=0 tx=0).stop 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406bbf0 msgr2=0x7fc40419ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406bbf0 0x7fc40419ff80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec000c00 tx=0x7fc3ec0047b0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 shutdown_connections 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc4041095c0 0x7fc404199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3f00b2990 0x7fc3f00b4e50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc40406b230 0x7fc40419fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 --2- 172.21.15.47:0/2773422011 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc40406bbf0 0x7fc40419ff80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 >> 172.21.15.47:0/2773422011 conn(0x7fc404079400 msgr2=0x7fc404101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 shutdown_connections 2024-05-11T12:22:54.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:54.853+0000 7fc40a265700 1 -- 172.21.15.47:0/2773422011 wait complete. 2024-05-11T12:22:54.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:54 smithi050 ceph-mon[34072]: pgmap v349: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:22:55.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:54 smithi047 ceph-mon[30375]: pgmap v349: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:22:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:54 smithi064 ceph-mon[34537]: pgmap v349: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:22:55.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953541 got 137438953546 for osd.4 2024-05-11T12:22:55.872 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:55.938 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:55 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2773422011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-11T12:22:55.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:55 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2773422011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-11T12:22:56.017 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:55 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2773422011' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-11T12:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.051+0000 7f6c49b4a700 1 -- 172.21.15.47:0/1097889286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406bc80 msgr2=0x7f6c4406c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.051+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/1097889286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406bc80 0x7f6c4406c100 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c38009990 tx=0x7f6c38009ca0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.051+0000 7f6c49b4a700 1 -- 172.21.15.47:0/1097889286 shutdown_connections 2024-05-11T12:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.051+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/1097889286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4406c640 0x7f6c441103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.051+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/1097889286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c441095c0 0x7f6c441099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.051+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/1097889286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406bc80 0x7f6c4406c100 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.051+0000 7f6c49b4a700 1 -- 172.21.15.47:0/1097889286 >> 172.21.15.47:0/1097889286 conn(0x7f6c44079400 msgr2=0x7f6c44079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.052+0000 7f6c49b4a700 1 -- 172.21.15.47:0/1097889286 shutdown_connections 2024-05-11T12:22:56.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.052+0000 7f6c49b4a700 1 -- 172.21.15.47:0/1097889286 wait complete. 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.052+0000 7f6c49b4a700 1 Processor -- start 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c49b4a700 1 -- start start 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c49b4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4406bc80 0x7f6c44102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c49b4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406c640 0x7f6c44102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c49b4a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c441095c0 0x7f6c44102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c49b4a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6c441136d0 con 0x7f6c4406c640 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c49b4a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44113550 con 0x7f6c441095c0 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c49b4a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44113850 con 0x7f6c4406bc80 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406c640 0x7f6c44102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406c640 0x7f6c44102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37078/0 (socket says 172.21.15.47:37078) 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 -- 172.21.15.47:0/796386668 learned_addr learned my addr 172.21.15.47:0/796386668 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:22:56.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c437fe700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4406bc80 0x7f6c44102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c43fff700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c441095c0 0x7f6c44102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 -- 172.21.15.47:0/796386668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4406bc80 msgr2=0x7f6c44102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4406bc80 0x7f6c44102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 -- 172.21.15.47:0/796386668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c441095c0 msgr2=0x7f6c44102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c441095c0 0x7f6c44102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.053+0000 7f6c42ffd700 1 -- 172.21.15.47:0/796386668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c441a8190 con 0x7f6c4406c640 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.054+0000 7f6c42ffd700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406c640 0x7f6c44102a90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000c00 tx=0x7f6c380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.054+0000 7f6c40ff9700 1 -- 172.21.15.47:0/796386668 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38027070 con 0x7f6c4406c640 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.054+0000 7f6c40ff9700 1 -- 172.21.15.47:0/796386668 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c38003f20 con 0x7f6c4406c640 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.054+0000 7f6c40ff9700 1 -- 172.21.15.47:0/796386668 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c38022ca0 con 0x7f6c4406c640 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.054+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c441a8420 con 0x7f6c4406c640 2024-05-11T12:22:56.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.054+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c441a8820 con 0x7f6c4406c640 2024-05-11T12:22:56.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.055+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4406a240 con 0x7f6c4406c640 2024-05-11T12:22:56.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.056+0000 7f6c40ff9700 1 -- 172.21.15.47:0/796386668 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6c3802a070 con 0x7f6c4406c640 2024-05-11T12:22:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.057+0000 7f6c40ff9700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6c2c0b2910 0x7f6c2c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:22:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.057+0000 7f6c437fe700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6c2c0b2910 0x7f6c2c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:22:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.057+0000 7f6c40ff9700 1 -- 172.21.15.47:0/796386668 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f6c380e39b0 con 0x7f6c4406c640 2024-05-11T12:22:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.058+0000 7f6c437fe700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6c2c0b2910 0x7f6c2c0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c34005950 tx=0x7f6c3400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:22:56.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.061+0000 7f6c40ff9700 1 -- 172.21.15.47:0/796386668 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c38031070 con 0x7f6c4406c640 2024-05-11T12:22:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.283+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6c4402e570 con 0x7f6c4406c640 2024-05-11T12:22:56.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.283+0000 7f6c40ff9700 1 -- 172.21.15.47:0/796386668 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6c38013070 con 0x7f6c4406c640 2024-05-11T12:22:56.283 INFO:teuthology.orchestra.run.smithi047.stdout:111669149775 2024-05-11T12:22:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6c2c0b2910 msgr2=0x7f6c2c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6c2c0b2910 0x7f6c2c0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6c34005950 tx=0x7f6c3400b410 comp rx=0 tx=0).stop 2024-05-11T12:22:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406c640 msgr2=0x7f6c44102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:22:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406c640 0x7f6c44102a90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c38000c00 tx=0x7f6c380047b0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 shutdown_connections 2024-05-11T12:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6c2c0b2910 0x7f6c2c0b4dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6c441095c0 0x7f6c44102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6c4406bc80 0x7f6c44102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 --2- 172.21.15.47:0/796386668 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c4406c640 0x7f6c44102a90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.285+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 >> 172.21.15.47:0/796386668 conn(0x7f6c44079400 msgr2=0x7f6c441012d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.286+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 shutdown_connections 2024-05-11T12:22:56.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:22:56.286+0000 7f6c49b4a700 1 -- 172.21.15.47:0/796386668 wait complete. 2024-05-11T12:22:56.831 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:56 smithi047 ceph-mon[30375]: pgmap v350: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:22:56.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:56 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/796386668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-11T12:22:56.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:56 smithi050 ceph-mon[34072]: pgmap v350: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:22:56.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:56 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/796386668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-11T12:22:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:56 smithi064 ceph-mon[34537]: pgmap v350: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:22:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:56 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/796386668' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-11T12:22:57.596 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149770 got 111669149775 for osd.3 2024-05-11T12:22:57.596 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:58.803 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593786 got 184683593791 for osd.6 2024-05-11T12:22:58.803 DEBUG:teuthology.parallel:result is None 2024-05-11T12:22:58.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:22:58 smithi050 ceph-mon[34072]: pgmap v351: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:59.016 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:22:58 smithi047 ceph-mon[30375]: pgmap v351: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:22:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:22:58 smithi064 ceph-mon[34537]: pgmap v351: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:00.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 -- 172.21.15.47:0/799872868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380108530 msgr2=0x7fc380108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 --2- 172.21.15.47:0/799872868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380108530 0x7fc380108910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc370009990 tx=0x7fc370009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 -- 172.21.15.47:0/799872868 shutdown_connections 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 --2- 172.21.15.47:0/799872868 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc380102e10 0x7fc38010f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 --2- 172.21.15.47:0/799872868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380108530 0x7fc380108910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 --2- 172.21.15.47:0/799872868 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc380102450 0x7fc3801028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 -- 172.21.15.47:0/799872868 >> 172.21.15.47:0/799872868 conn(0x7fc380079400 msgr2=0x7fc380079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 -- 172.21.15.47:0/799872868 shutdown_connections 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.108+0000 7fc38626d700 1 -- 172.21.15.47:0/799872868 wait complete. 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 Processor -- start 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 -- start start 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380102450 0x7fc38019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc380102e10 0x7fc38019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc380108530 0x7fc380199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc380112680 con 0x7fc380102450 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc380112500 con 0x7fc380102e10 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.109+0000 7fc38626d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc380112800 con 0x7fc380108530 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380102450 0x7fc38019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380102450 0x7fc38019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54784/0 (socket says 172.21.15.47:54784) 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 -- 172.21.15.47:0/2661450811 learned_addr learned my addr 172.21.15.47:0/2661450811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc38480a700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc380108530 0x7fc380199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37f7fe700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc380102e10 0x7fc38019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 -- 172.21.15.47:0/2661450811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc380108530 msgr2=0x7fc380199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc380108530 0x7fc380199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 -- 172.21.15.47:0/2661450811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc380102e10 msgr2=0x7fc38019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc380102e10 0x7fc38019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 -- 172.21.15.47:0/2661450811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38019a1f0 con 0x7fc380102450 2024-05-11T12:23:00.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37ffff700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380102450 0x7fc38019f8d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc370000c00 tx=0x7fc3700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:00.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37d7fa700 1 -- 172.21.15.47:0/2661450811 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370027070 con 0x7fc380102450 2024-05-11T12:23:00.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37d7fa700 1 -- 172.21.15.47:0/2661450811 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc370003fa0 con 0x7fc380102450 2024-05-11T12:23:00.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38019a480 con 0x7fc380102450 2024-05-11T12:23:00.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc37d7fa700 1 -- 172.21.15.47:0/2661450811 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370022db0 con 0x7fc380102450 2024-05-11T12:23:00.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.110+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3801a67a0 con 0x7fc380102450 2024-05-11T12:23:00.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.111+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc38006a240 con 0x7fc380102450 2024-05-11T12:23:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.113+0000 7fc37d7fa700 1 -- 172.21.15.47:0/2661450811 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc37002a070 con 0x7fc380102450 2024-05-11T12:23:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.113+0000 7fc37d7fa700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3680b2910 0x7fc3680b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.114+0000 7fc37f7fe700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3680b2910 0x7fc3680b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.114+0000 7fc37d7fa700 1 -- 172.21.15.47:0/2661450811 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc3700e3530 con 0x7fc380102450 2024-05-11T12:23:00.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.114+0000 7fc37f7fe700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3680b2910 0x7fc3680b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc374005950 tx=0x7fc3740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:00.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.117+0000 7fc37d7fa700 1 -- 172.21.15.47:0/2661450811 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc370013070 con 0x7fc380102450 2024-05-11T12:23:00.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.336+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc3801a6970 con 0x7fc380102450 2024-05-11T12:23:00.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.336+0000 7fc37d7fa700 1 -- 172.21.15.47:0/2661450811 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc3700ab0b0 con 0x7fc380102450 2024-05-11T12:23:00.336 INFO:teuthology.orchestra.run.smithi047.stdout:34359738462 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3680b2910 msgr2=0x7fc3680b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3680b2910 0x7fc3680b4dd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc374005950 tx=0x7fc3740058e0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380102450 msgr2=0x7fc38019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380102450 0x7fc38019f8d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc370000c00 tx=0x7fc3700047b0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 shutdown_connections 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc380108530 0x7fc380199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc380102450 0x7fc38019f8d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc3680b2910 0x7fc3680b4dd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 --2- 172.21.15.47:0/2661450811 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc380102e10 0x7fc38019fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 >> 172.21.15.47:0/2661450811 conn(0x7fc380079400 msgr2=0x7fc38006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 shutdown_connections 2024-05-11T12:23:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.339+0000 7fc38626d700 1 -- 172.21.15.47:0/2661450811 wait complete. 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 -- 172.21.15.47:0/2891744691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 msgr2=0x7ff23c1104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 --2- 172.21.15.47:0/2891744691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c1104a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff23400b0d0 tx=0x7ff23400b3e0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 -- 172.21.15.47:0/2891744691 shutdown_connections 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 --2- 172.21.15.47:0/2891744691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c1104a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 --2- 172.21.15.47:0/2891744691 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff23c103830 0x7ff23c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 --2- 172.21.15.47:0/2891744691 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c1041e0 0x7ff23c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 -- 172.21.15.47:0/2891744691 >> 172.21.15.47:0/2891744691 conn(0x7ff23c07a6c0 msgr2=0x7ff23c07aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:00.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 -- 172.21.15.47:0/2891744691 shutdown_connections 2024-05-11T12:23:00.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.565+0000 7ff242aaf700 1 -- 172.21.15.47:0/2891744691 wait complete. 2024-05-11T12:23:00.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 Processor -- start 2024-05-11T12:23:00.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 -- start start 2024-05-11T12:23:00.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c103830 0x7ff23c075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff23c1041e0 0x7ff23c075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c0762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c113710 con 0x7ff23c108920 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c113590 con 0x7ff23c103830 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff242aaf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff23c113890 con 0x7ff23c1041e0 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff24104c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c0762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff23bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff23c1041e0 0x7ff23c075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff24104c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c0762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54804/0 (socket says 172.21.15.47:54804) 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff24084b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c103830 0x7ff23c075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff24104c700 1 -- 172.21.15.47:0/3612101511 learned_addr learned my addr 172.21.15.47:0/3612101511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff23bfff700 1 -- 172.21.15.47:0/3612101511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c103830 msgr2=0x7ff23c075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff23bfff700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c103830 0x7ff23c075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff23bfff700 1 -- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 msgr2=0x7ff23c0762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff23bfff700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c0762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff23bfff700 1 -- 172.21.15.47:0/3612101511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c1a8010 con 0x7ff23c1041e0 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.566+0000 7ff24104c700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c0762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.567+0000 7ff23bfff700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff23c1041e0 0x7ff23c075d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff23000e9f0 tx=0x7ff23000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.567+0000 7ff239ffb700 1 -- 172.21.15.47:0/3612101511 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23000c9c0 con 0x7ff23c1041e0 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.567+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff23c1a8300 con 0x7ff23c1041e0 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.567+0000 7ff239ffb700 1 -- 172.21.15.47:0/3612101511 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff23000cb60 con 0x7ff23c1041e0 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.567+0000 7ff239ffb700 1 -- 172.21.15.47:0/3612101511 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230018680 con 0x7ff23c1041e0 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.567+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff23c1a8840 con 0x7ff23c1041e0 2024-05-11T12:23:00.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.568+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff23c10baf0 con 0x7ff23c1041e0 2024-05-11T12:23:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.570+0000 7ff239ffb700 1 -- 172.21.15.47:0/3612101511 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff230010430 con 0x7ff23c1041e0 2024-05-11T12:23:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.571+0000 7ff239ffb700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2240b2910 0x7ff2240b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.571+0000 7ff24084b700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2240b2910 0x7ff2240b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.571+0000 7ff239ffb700 1 -- 172.21.15.47:0/3612101511 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff2300e2930 con 0x7ff23c1041e0 2024-05-11T12:23:00.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.571+0000 7ff24084b700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2240b2910 0x7ff2240b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009c60 tx=0x7ff22c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:00.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.575+0000 7ff239ffb700 1 -- 172.21.15.47:0/3612101511 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2300aa3c0 con 0x7ff23c1041e0 2024-05-11T12:23:00.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.793+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff23c10b4d0 con 0x7ff23c1041e0 2024-05-11T12:23:00.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.794+0000 7ff239ffb700 1 -- 172.21.15.47:0/3612101511 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff2300a9cc0 con 0x7ff23c1041e0 2024-05-11T12:23:00.794 INFO:teuthology.orchestra.run.smithi047.stdout:77309411413 2024-05-11T12:23:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2240b2910 msgr2=0x7ff2240b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2240b2910 0x7ff2240b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009c60 tx=0x7ff22c009260 comp rx=0 tx=0).stop 2024-05-11T12:23:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff23c1041e0 msgr2=0x7ff23c075d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff23c1041e0 0x7ff23c075d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff23000e9f0 tx=0x7ff23000ed00 comp rx=0 tx=0).stop 2024-05-11T12:23:00.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 shutdown_connections 2024-05-11T12:23:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff23c108920 0x7ff23c0762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2240b2910 0x7ff2240b4dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff23c103830 0x7ff23c075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 --2- 172.21.15.47:0/3612101511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff23c1041e0 0x7ff23c075d90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.797+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 >> 172.21.15.47:0/3612101511 conn(0x7ff23c07a6c0 msgr2=0x7ff23c102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.798+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 shutdown_connections 2024-05-11T12:23:00.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:00.798+0000 7ff242aaf700 1 -- 172.21.15.47:0/3612101511 wait complete. 2024-05-11T12:23:00.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:00 smithi047 ceph-mon[30375]: pgmap v352: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:00.904 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:00 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2661450811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-11T12:23:00.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:00 smithi050 ceph-mon[34072]: pgmap v352: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:00.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:00 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2661450811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-11T12:23:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:00 smithi064 ceph-mon[34537]: pgmap v352: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:01.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:00 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2661450811' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-11T12:23:01.217 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738462 for osd.0 2024-05-11T12:23:01.217 DEBUG:teuthology.parallel:result is None 2024-05-11T12:23:01.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411413 for osd.2 2024-05-11T12:23:01.890 DEBUG:teuthology.parallel:result is None 2024-05-11T12:23:01.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:01 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3612101511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-11T12:23:02.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:01 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3612101511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-11T12:23:02.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:01 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3612101511' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-11T12:23:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 -- 172.21.15.47:0/4087955235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29581026b0 msgr2=0x7f2958108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 --2- 172.21.15.47:0/4087955235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29581026b0 0x7f2958108210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f294c0099c0 tx=0x7f294c009cd0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 -- 172.21.15.47:0/4087955235 shutdown_connections 2024-05-11T12:23:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 --2- 172.21.15.47:0/4087955235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2958108940 0x7f29581104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 --2- 172.21.15.47:0/4087955235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2958101d00 0x7f29581020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 --2- 172.21.15.47:0/4087955235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29581026b0 0x7f2958108210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 -- 172.21.15.47:0/4087955235 >> 172.21.15.47:0/4087955235 conn(0x7f2958078a50 msgr2=0x7f29580ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:02.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 -- 172.21.15.47:0/4087955235 shutdown_connections 2024-05-11T12:23:02.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.357+0000 7f295c9c0700 1 -- 172.21.15.47:0/4087955235 wait complete. 2024-05-11T12:23:02.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 Processor -- start 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 -- start start 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2958101d00 0x7f295819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29581026b0 0x7f295819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2958108940 0x7f29581999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2958113720 con 0x7f2958101d00 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f29581135a0 con 0x7f29581026b0 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295c9c0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29581138a0 con 0x7f2958108940 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2958101d00 0x7f295819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2958101d00 0x7f295819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54812/0 (socket says 172.21.15.47:54812) 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.358+0000 7f295659c700 1 -- 172.21.15.47:0/797495149 learned_addr learned my addr 172.21.15.47:0/797495149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295659c700 1 -- 172.21.15.47:0/797495149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2958108940 msgr2=0x7f29581999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295659c700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2958108940 0x7f29581999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295659c700 1 -- 172.21.15.47:0/797495149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29581026b0 msgr2=0x7f295819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f2955d9b700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29581026b0 0x7f295819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295659c700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29581026b0 0x7f295819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295659c700 1 -- 172.21.15.47:0/797495149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295819a120 con 0x7f2958101d00 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f2955d9b700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29581026b0 0x7f295819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295659c700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2958101d00 0x7f295819f7b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f294800ada0 tx=0x7f294800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f29477fe700 1 -- 172.21.15.47:0/797495149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948017020 con 0x7f2958101d00 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f29477fe700 1 -- 172.21.15.47:0/797495149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f294800ce30 con 0x7f2958101d00 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295819a410 con 0x7f2958101d00 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f29477fe700 1 -- 172.21.15.47:0/797495149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29480055a0 con 0x7f2958101d00 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.359+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29581a6680 con 0x7f2958101d00 2024-05-11T12:23:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.360+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295806a240 con 0x7f2958101d00 2024-05-11T12:23:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.361+0000 7f29477fe700 1 -- 172.21.15.47:0/797495149 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2948004c40 con 0x7f2958101d00 2024-05-11T12:23:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.362+0000 7f29477fe700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29400b2910 0x7f29400b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.362+0000 7f2955d9b700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29400b2910 0x7f29400b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.362+0000 7f29477fe700 1 -- 172.21.15.47:0/797495149 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f29480e2300 con 0x7f2958101d00 2024-05-11T12:23:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.363+0000 7f2955d9b700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29400b2910 0x7f29400b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f294c003800 tx=0x7f294c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:02.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.366+0000 7f29477fe700 1 -- 172.21.15.47:0/797495149 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2948021020 con 0x7f2958101d00 2024-05-11T12:23:02.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.581+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f295802e570 con 0x7f2958101d00 2024-05-11T12:23:02.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.582+0000 7f29477fe700 1 -- 172.21.15.47:0/797495149 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f29480a9e20 con 0x7f2958101d00 2024-05-11T12:23:02.582 INFO:teuthology.orchestra.run.smithi047.stdout:55834574938 2024-05-11T12:23:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29400b2910 msgr2=0x7f29400b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29400b2910 0x7f29400b4dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f294c003800 tx=0x7f294c009f10 comp rx=0 tx=0).stop 2024-05-11T12:23:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2958101d00 msgr2=0x7f295819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2958101d00 0x7f295819f7b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f294800ada0 tx=0x7f294800c330 comp rx=0 tx=0).stop 2024-05-11T12:23:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 shutdown_connections 2024-05-11T12:23:02.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2958108940 0x7f29581999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2958101d00 0x7f295819f7b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29400b2910 0x7f29400b4dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 --2- 172.21.15.47:0/797495149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29581026b0 0x7f295819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 >> 172.21.15.47:0/797495149 conn(0x7f2958078a50 msgr2=0x7f29581070f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 shutdown_connections 2024-05-11T12:23:02.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:02.584+0000 7f295c9c0700 1 -- 172.21.15.47:0/797495149 wait complete. 2024-05-11T12:23:02.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:02 smithi050 ceph-mon[34072]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:02.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:02 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/797495149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-11T12:23:02.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:02 smithi047 ceph-mon[30375]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:02.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:02 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/797495149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-11T12:23:03.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:02 smithi064 ceph-mon[34537]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:03.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:02 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/797495149' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-11T12:23:03.249 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574934 got 55834574938 for osd.1 2024-05-11T12:23:03.249 DEBUG:teuthology.parallel:result is None 2024-05-11T12:23:03.250 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-11T12:23:03.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T12:23:03.506 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:04.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:04 smithi050 ceph-mon[34072]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:04.975 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:04 smithi047 ceph-mon[30375]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:05.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:04 smithi064 ceph-mon[34537]: pgmap v354: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 -- 172.21.15.47:0/985782965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 msgr2=0x7fa6281103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 --2- 172.21.15.47:0/985782965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa6281103d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa620009990 tx=0x7fa620009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 -- 172.21.15.47:0/985782965 shutdown_connections 2024-05-11T12:23:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 --2- 172.21.15.47:0/985782965 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa6281103d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 --2- 172.21.15.47:0/985782965 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa6281095b0 0x7fa628109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 --2- 172.21.15.47:0/985782965 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa62806bac0 0x7fa62806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 -- 172.21.15.47:0/985782965 >> 172.21.15.47:0/985782965 conn(0x7fa6280793c0 msgr2=0x7fa6280797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 -- 172.21.15.47:0/985782965 shutdown_connections 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.216+0000 7fa63052d700 1 -- 172.21.15.47:0/985782965 wait complete. 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 Processor -- start 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 -- start start 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa62806bac0 0x7fa62819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa62819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6281095b0 0x7fa628199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281136f0 con 0x7fa62806c480 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113570 con 0x7fa62806bac0 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa63052d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa628113870 con 0x7fa6281095b0 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.217+0000 7fa62dac8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa62819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa62819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54836/0 (socket says 172.21.15.47:54836) 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 -- 172.21.15.47:0/3374859933 learned_addr learned my addr 172.21.15.47:0/3374859933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62eaca700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6281095b0 0x7fa628199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62e2c9700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa62806bac0 0x7fa62819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 -- 172.21.15.47:0/3374859933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6281095b0 msgr2=0x7fa628199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6281095b0 0x7fa628199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 -- 172.21.15.47:0/3374859933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa62806bac0 msgr2=0x7fa62819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa62806bac0 0x7fa62819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 -- 172.21.15.47:0/3374859933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819a1b0 con 0x7fa62806c480 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62eaca700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6281095b0 0x7fa628199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa62dac8700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa62819fdd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c970 tx=0x7fa61c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa6277fe700 1 -- 172.21.15.47:0/3374859933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c007aa0 con 0x7fa62806c480 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62819a4a0 con 0x7fa62806c480 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa6277fe700 1 -- 172.21.15.47:0/3374859933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c004ce0 con 0x7fa62806c480 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa6277fe700 1 -- 172.21.15.47:0/3374859933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0055c0 con 0x7fa62806c480 2024-05-11T12:23:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.218+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6281a6760 con 0x7fa62806c480 2024-05-11T12:23:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.219+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62806a240 con 0x7fa62806c480 2024-05-11T12:23:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.221+0000 7fa6277fe700 1 -- 172.21.15.47:0/3374859933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa61c007c40 con 0x7fa62806c480 2024-05-11T12:23:05.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.221+0000 7fa6277fe700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa6100b2910 0x7fa6100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:05.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.221+0000 7fa62e2c9700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa6100b2910 0x7fa6100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:05.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.221+0000 7fa6277fe700 1 -- 172.21.15.47:0/3374859933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fa61c0e25b0 con 0x7fa62806c480 2024-05-11T12:23:05.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.222+0000 7fa62e2c9700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa6100b2910 0x7fa6100b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa618007700 tx=0x7fa618007690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:05.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.225+0000 7fa6277fe700 1 -- 172.21.15.47:0/3374859933 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa61c0aa1c0 con 0x7fa62806c480 2024-05-11T12:23:05.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.436+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa628063990 con 0x7fa6100b2910 2024-05-11T12:23:05.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.444+0000 7fa6277fe700 1 -- 172.21.15.47:0/3374859933 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7fa628063990 con 0x7fa6100b2910 2024-05-11T12:23:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa6100b2910 msgr2=0x7fa6100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa6100b2910 0x7fa6100b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa618007700 tx=0x7fa618007690 comp rx=0 tx=0).stop 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 msgr2=0x7fa62819fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa62819fdd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c970 tx=0x7fa61c00cd30 comp rx=0 tx=0).stop 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 shutdown_connections 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa6281095b0 0x7fa628199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa6100b2910 0x7fa6100b4dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa62806bac0 0x7fa62819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 --2- 172.21.15.47:0/3374859933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa62806c480 0x7fa62819fdd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 >> 172.21.15.47:0/3374859933 conn(0x7fa6280793c0 msgr2=0x7fa6281011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 shutdown_connections 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:05.447+0000 7fa63052d700 1 -- 172.21.15.47:0/3374859933 wait complete. 2024-05-11T12:23:05.450 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-05-11T12:23:06.085 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-05-11T12:23:03.559113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653150,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":275,"num_read_kb":226,"num_write":210,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951874,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393400,"kb_used_data":10896,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124548424,"statfs":{"total":1151940427776,"available":1151537586176,"internally_reserved":0,"allocated":11157504,"data_stored":7302630,"data_compressed":246459,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002817"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651849+0000","last_change":"2024-05-11T12:21:11.536657+0000","last_active":"2024-05-11T12:22:43.651849+0000","last_peered":"2024-05-11T12:22:43.651849+0000","last_clean":"2024-05-11T12:22:43.651849+0000","last_became_active":"2024-05-11T12:21:11.536243+0000","last_became_peered":"2024-05-11T12:21:11.536243+0000","last_unstale":"2024-05-11T12:22:43.651849+0000","last_undegraded":"2024-05-11T12:22:43.651849+0000","last_fullsized":"2024-05-11T12:22:43.651849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:29:20.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650830+0000","last_change":"2024-05-11T12:21:11.527602+0000","last_active":"2024-05-11T12:22:43.650830+0000","last_peered":"2024-05-11T12:22:43.650830+0000","last_clean":"2024-05-11T12:22:43.650830+0000","last_became_active":"2024-05-11T12:21:11.527309+0000","last_became_peered":"2024-05-11T12:21:11.527309+0000","last_unstale":"2024-05-11T12:22:43.650830+0000","last_undegraded":"2024-05-11T12:22:43.650830+0000","last_fullsized":"2024-05-11T12:22:43.650830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:03.764570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650532+0000","last_change":"2024-05-11T12:21:11.529094+0000","last_active":"2024-05-11T12:22:43.650532+0000","last_peered":"2024-05-11T12:22:43.650532+0000","last_clean":"2024-05-11T12:22:43.650532+0000","last_became_active":"2024-05-11T12:21:11.528796+0000","last_became_peered":"2024-05-11T12:21:11.528796+0000","last_unstale":"2024-05-11T12:22:43.650532+0000","last_undegraded":"2024-05-11T12:22:43.650532+0000","last_fullsized":"2024-05-11T12:22:43.650532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:46:41.098066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648539+0000","last_change":"2024-05-11T12:21:11.528903+0000","last_active":"2024-05-11T12:22:43.648539+0000","last_peered":"2024-05-11T12:22:43.648539+0000","last_clean":"2024-05-11T12:22:43.648539+0000","last_became_active":"2024-05-11T12:21:11.528545+0000","last_became_peered":"2024-05-11T12:21:11.528545+0000","last_unstale":"2024-05-11T12:22:43.648539+0000","last_undegraded":"2024-05-11T12:22:43.648539+0000","last_fullsized":"2024-05-11T12:22:43.648539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:56.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093747+0000","last_change":"2024-05-11T12:21:11.521074+0000","last_active":"2024-05-11T12:22:44.093747+0000","last_peered":"2024-05-11T12:22:44.093747+0000","last_clean":"2024-05-11T12:22:44.093747+0000","last_became_active":"2024-05-11T12:21:11.520675+0000","last_became_peered":"2024-05-11T12:21:11.520675+0000","last_unstale":"2024-05-11T12:22:44.093747+0000","last_undegraded":"2024-05-11T12:22:44.093747+0000","last_fullsized":"2024-05-11T12:22:44.093747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:22:50.952687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748814+0000","last_change":"2024-05-11T12:21:42.671243+0000","last_active":"2024-05-11T12:22:43.748814+0000","last_peered":"2024-05-11T12:22:43.748814+0000","last_clean":"2024-05-11T12:22:43.748814+0000","last_became_active":"2024-05-11T12:21:42.670801+0000","last_became_peered":"2024-05-11T12:21:42.670801+0000","last_unstale":"2024-05-11T12:22:43.748814+0000","last_undegraded":"2024-05-11T12:22:43.748814+0000","last_fullsized":"2024-05-11T12:22:43.748814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:02.539855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.095706+0000","last_change":"2024-05-11T12:22:44.095706+0000","last_active":"2024-05-11T12:22:44.095706+0000","last_peered":"2024-05-11T12:22:44.095706+0000","last_clean":"2024-05-11T12:22:44.095706+0000","last_became_active":"2024-05-11T12:22:44.095177+0000","last_became_peered":"2024-05-11T12:22:44.095177+0000","last_unstale":"2024-05-11T12:22:44.095706+0000","last_undegraded":"2024-05-11T12:22:44.095706+0000","last_fullsized":"2024-05-11T12:22:44.095706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:11.426607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.789981+0000","last_change":"2024-05-11T12:21:11.532874+0000","last_active":"2024-05-11T12:22:43.789981+0000","last_peered":"2024-05-11T12:22:43.789981+0000","last_clean":"2024-05-11T12:22:43.789981+0000","last_became_active":"2024-05-11T12:21:11.532237+0000","last_became_peered":"2024-05-11T12:21:11.532237+0000","last_unstale":"2024-05-11T12:22:43.789981+0000","last_undegraded":"2024-05-11T12:22:43.789981+0000","last_fullsized":"2024-05-11T12:22:43.789981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:13.770719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.892174+0000","last_change":"2024-05-11T12:21:11.536465+0000","last_active":"2024-05-11T12:22:43.892174+0000","last_peered":"2024-05-11T12:22:43.892174+0000","last_clean":"2024-05-11T12:22:43.892174+0000","last_became_active":"2024-05-11T12:21:11.536057+0000","last_became_peered":"2024-05-11T12:21:11.536057+0000","last_unstale":"2024-05-11T12:22:43.892174+0000","last_undegraded":"2024-05-11T12:22:43.892174+0000","last_fullsized":"2024-05-11T12:22:43.892174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:22:45.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.749219+0000","last_change":"2024-05-11T12:21:11.528187+0000","last_active":"2024-05-11T12:22:43.749219+0000","last_peered":"2024-05-11T12:22:43.749219+0000","last_clean":"2024-05-11T12:22:43.749219+0000","last_became_active":"2024-05-11T12:21:11.527780+0000","last_became_peered":"2024-05-11T12:21:11.527780+0000","last_unstale":"2024-05-11T12:22:43.749219+0000","last_undegraded":"2024-05-11T12:22:43.749219+0000","last_fullsized":"2024-05-11T12:22:43.749219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:23.484372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649613+0000","last_change":"2024-05-11T12:21:11.543388+0000","last_active":"2024-05-11T12:22:43.649613+0000","last_peered":"2024-05-11T12:22:43.649613+0000","last_clean":"2024-05-11T12:22:43.649613+0000","last_became_active":"2024-05-11T12:21:11.542984+0000","last_became_peered":"2024-05-11T12:21:11.542984+0000","last_unstale":"2024-05-11T12:22:43.649613+0000","last_undegraded":"2024-05-11T12:22:43.649613+0000","last_fullsized":"2024-05-11T12:22:43.649613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:52.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648390+0000","last_change":"2024-05-11T12:21:11.523355+0000","last_active":"2024-05-11T12:22:43.648390+0000","last_peered":"2024-05-11T12:22:43.648390+0000","last_clean":"2024-05-11T12:22:43.648390+0000","last_became_active":"2024-05-11T12:21:11.523007+0000","last_became_peered":"2024-05-11T12:21:11.523007+0000","last_unstale":"2024-05-11T12:22:43.648390+0000","last_undegraded":"2024-05-11T12:22:43.648390+0000","last_fullsized":"2024-05-11T12:22:43.648390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:57.197028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844877+0000","last_change":"2024-05-11T12:21:11.528072+0000","last_active":"2024-05-11T12:22:43.844877+0000","last_peered":"2024-05-11T12:22:43.844877+0000","last_clean":"2024-05-11T12:22:43.844877+0000","last_became_active":"2024-05-11T12:21:11.527478+0000","last_became_peered":"2024-05-11T12:21:11.527478+0000","last_unstale":"2024-05-11T12:22:43.844877+0000","last_undegraded":"2024-05-11T12:22:43.844877+0000","last_fullsized":"2024-05-11T12:22:43.844877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:57:29.001602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650344+0000","last_change":"2024-05-11T12:21:11.526862+0000","last_active":"2024-05-11T12:22:43.650344+0000","last_peered":"2024-05-11T12:22:43.650344+0000","last_clean":"2024-05-11T12:22:43.650344+0000","last_became_active":"2024-05-11T12:21:11.526553+0000","last_became_peered":"2024-05-11T12:21:11.526553+0000","last_unstale":"2024-05-11T12:22:43.650344+0000","last_undegraded":"2024-05-11T12:22:43.650344+0000","last_fullsized":"2024-05-11T12:22:43.650344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:48.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650178+0000","last_change":"2024-05-11T12:21:42.669393+0000","last_active":"2024-05-11T12:22:43.650178+0000","last_peered":"2024-05-11T12:22:43.650178+0000","last_clean":"2024-05-11T12:22:43.650178+0000","last_became_active":"2024-05-11T12:21:42.669000+0000","last_became_peered":"2024-05-11T12:21:42.669000+0000","last_unstale":"2024-05-11T12:22:43.650178+0000","last_undegraded":"2024-05-11T12:22:43.650178+0000","last_fullsized":"2024-05-11T12:22:43.650178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:29.439588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891535+0000","last_change":"2024-05-11T12:21:11.540698+0000","last_active":"2024-05-11T12:22:43.891535+0000","last_peered":"2024-05-11T12:22:43.891535+0000","last_clean":"2024-05-11T12:22:43.891535+0000","last_became_active":"2024-05-11T12:21:11.539899+0000","last_became_peered":"2024-05-11T12:21:11.539899+0000","last_unstale":"2024-05-11T12:22:43.891535+0000","last_undegraded":"2024-05-11T12:22:43.891535+0000","last_fullsized":"2024-05-11T12:22:43.891535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:48.222664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651942+0000","last_change":"2024-05-11T12:21:11.537055+0000","last_active":"2024-05-11T12:22:43.651942+0000","last_peered":"2024-05-11T12:22:43.651942+0000","last_clean":"2024-05-11T12:22:43.651942+0000","last_became_active":"2024-05-11T12:21:11.536625+0000","last_became_peered":"2024-05-11T12:21:11.536625+0000","last_unstale":"2024-05-11T12:22:43.651942+0000","last_undegraded":"2024-05-11T12:22:43.651942+0000","last_fullsized":"2024-05-11T12:22:43.651942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:33.453467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647256+0000","last_change":"2024-05-11T12:21:11.530254+0000","last_active":"2024-05-11T12:22:43.647256+0000","last_peered":"2024-05-11T12:22:43.647256+0000","last_clean":"2024-05-11T12:22:43.647256+0000","last_became_active":"2024-05-11T12:21:11.529890+0000","last_became_peered":"2024-05-11T12:21:11.529890+0000","last_unstale":"2024-05-11T12:22:43.647256+0000","last_undegraded":"2024-05-11T12:22:43.647256+0000","last_fullsized":"2024-05-11T12:22:43.647256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:36.100931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653657+0000","last_change":"2024-05-11T12:21:11.532893+0000","last_active":"2024-05-11T12:22:43.653657+0000","last_peered":"2024-05-11T12:22:43.653657+0000","last_clean":"2024-05-11T12:22:43.653657+0000","last_became_active":"2024-05-11T12:21:11.532539+0000","last_became_peered":"2024-05-11T12:21:11.532539+0000","last_unstale":"2024-05-11T12:22:43.653657+0000","last_undegraded":"2024-05-11T12:22:43.653657+0000","last_fullsized":"2024-05-11T12:22:43.653657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:14:57.888102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647499+0000","last_change":"2024-05-11T12:21:08.510909+0000","last_active":"2024-05-11T12:22:43.647499+0000","last_peered":"2024-05-11T12:22:43.647499+0000","last_clean":"2024-05-11T12:22:43.647499+0000","last_became_active":"2024-05-11T12:21:08.509986+0000","last_became_peered":"2024-05-11T12:21:08.509986+0000","last_unstale":"2024-05-11T12:22:43.647499+0000","last_undegraded":"2024-05-11T12:22:43.647499+0000","last_fullsized":"2024-05-11T12:22:43.647499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:03:36.430087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790218+0000","last_change":"2024-05-11T12:21:11.532403+0000","last_active":"2024-05-11T12:22:43.790218+0000","last_peered":"2024-05-11T12:22:43.790218+0000","last_clean":"2024-05-11T12:22:43.790218+0000","last_became_active":"2024-05-11T12:21:11.531907+0000","last_became_peered":"2024-05-11T12:21:11.531907+0000","last_unstale":"2024-05-11T12:22:43.790218+0000","last_undegraded":"2024-05-11T12:22:43.790218+0000","last_fullsized":"2024-05-11T12:22:43.790218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:53:27.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790424+0000","last_change":"2024-05-11T12:21:08.531664+0000","last_active":"2024-05-11T12:22:43.790424+0000","last_peered":"2024-05-11T12:22:43.790424+0000","last_clean":"2024-05-11T12:22:43.790424+0000","last_became_active":"2024-05-11T12:21:08.531319+0000","last_became_peered":"2024-05-11T12:21:08.531319+0000","last_unstale":"2024-05-11T12:22:43.790424+0000","last_undegraded":"2024-05-11T12:22:43.790424+0000","last_fullsized":"2024-05-11T12:22:43.790424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:16:08.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650677+0000","last_change":"2024-05-11T12:21:11.532132+0000","last_active":"2024-05-11T12:22:43.650677+0000","last_peered":"2024-05-11T12:22:43.650677+0000","last_clean":"2024-05-11T12:22:43.650677+0000","last_became_active":"2024-05-11T12:21:11.529580+0000","last_became_peered":"2024-05-11T12:21:11.529580+0000","last_unstale":"2024-05-11T12:22:43.650677+0000","last_undegraded":"2024-05-11T12:22:43.650677+0000","last_fullsized":"2024-05-11T12:22:43.650677+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:29.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094338+0000","last_change":"2024-05-11T12:21:42.674898+0000","last_active":"2024-05-11T12:22:44.094338+0000","last_peered":"2024-05-11T12:22:44.094338+0000","last_clean":"2024-05-11T12:22:44.094338+0000","last_became_active":"2024-05-11T12:21:42.669853+0000","last_became_peered":"2024-05-11T12:21:42.669853+0000","last_unstale":"2024-05-11T12:22:44.094338+0000","last_undegraded":"2024-05-11T12:22:44.094338+0000","last_fullsized":"2024-05-11T12:22:44.094338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:36.040786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891618+0000","last_change":"2024-05-11T12:21:11.534706+0000","last_active":"2024-05-11T12:22:43.891618+0000","last_peered":"2024-05-11T12:22:43.891618+0000","last_clean":"2024-05-11T12:22:43.891618+0000","last_became_active":"2024-05-11T12:21:11.534301+0000","last_became_peered":"2024-05-11T12:21:11.534301+0000","last_unstale":"2024-05-11T12:22:43.891618+0000","last_undegraded":"2024-05-11T12:22:43.891618+0000","last_fullsized":"2024-05-11T12:22:43.891618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:24.277988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093993+0000","last_change":"2024-05-11T12:21:08.513794+0000","last_active":"2024-05-11T12:22:44.093993+0000","last_peered":"2024-05-11T12:22:44.093993+0000","last_clean":"2024-05-11T12:22:44.093993+0000","last_became_active":"2024-05-11T12:21:08.513383+0000","last_became_peered":"2024-05-11T12:21:08.513383+0000","last_unstale":"2024-05-11T12:22:44.093993+0000","last_undegraded":"2024-05-11T12:22:44.093993+0000","last_fullsized":"2024-05-11T12:22:44.093993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:50.558886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751890+0000","last_change":"2024-05-11T12:21:11.537147+0000","last_active":"2024-05-11T12:22:43.751890+0000","last_peered":"2024-05-11T12:22:43.751890+0000","last_clean":"2024-05-11T12:22:43.751890+0000","last_became_active":"2024-05-11T12:21:11.536739+0000","last_became_peered":"2024-05-11T12:21:11.536739+0000","last_unstale":"2024-05-11T12:22:43.751890+0000","last_undegraded":"2024-05-11T12:22:43.751890+0000","last_fullsized":"2024-05-11T12:22:43.751890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:00:45.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751556+0000","last_change":"2024-05-11T12:21:08.513494+0000","last_active":"2024-05-11T12:22:43.751556+0000","last_peered":"2024-05-11T12:22:43.751556+0000","last_clean":"2024-05-11T12:22:43.751556+0000","last_became_active":"2024-05-11T12:21:08.513089+0000","last_became_peered":"2024-05-11T12:21:08.513089+0000","last_unstale":"2024-05-11T12:22:43.751556+0000","last_undegraded":"2024-05-11T12:22:43.751556+0000","last_fullsized":"2024-05-11T12:22:43.751556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:48:28.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093686+0000","last_change":"2024-05-11T12:21:11.527827+0000","last_active":"2024-05-11T12:22:44.093686+0000","last_peered":"2024-05-11T12:22:44.093686+0000","last_clean":"2024-05-11T12:22:44.093686+0000","last_became_active":"2024-05-11T12:21:11.527397+0000","last_became_peered":"2024-05-11T12:21:11.527397+0000","last_unstale":"2024-05-11T12:22:44.093686+0000","last_undegraded":"2024-05-11T12:22:44.093686+0000","last_fullsized":"2024-05-11T12:22:44.093686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:18.710855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748713+0000","last_change":"2024-05-11T12:21:08.513519+0000","last_active":"2024-05-11T12:22:43.748713+0000","last_peered":"2024-05-11T12:22:43.748713+0000","last_clean":"2024-05-11T12:22:43.748713+0000","last_became_active":"2024-05-11T12:21:08.512566+0000","last_became_peered":"2024-05-11T12:21:08.512566+0000","last_unstale":"2024-05-11T12:22:43.748713+0000","last_undegraded":"2024-05-11T12:22:43.748713+0000","last_fullsized":"2024-05-11T12:22:43.748713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:56.104043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751737+0000","last_change":"2024-05-11T12:21:11.535364+0000","last_active":"2024-05-11T12:22:43.751737+0000","last_peered":"2024-05-11T12:22:43.751737+0000","last_clean":"2024-05-11T12:22:43.751737+0000","last_became_active":"2024-05-11T12:21:11.534988+0000","last_became_peered":"2024-05-11T12:21:11.534988+0000","last_unstale":"2024-05-11T12:22:43.751737+0000","last_undegraded":"2024-05-11T12:22:43.751737+0000","last_fullsized":"2024-05-11T12:22:43.751737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:25:20.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844702+0000","last_change":"2024-05-11T12:21:08.509913+0000","last_active":"2024-05-11T12:22:43.844702+0000","last_peered":"2024-05-11T12:22:43.844702+0000","last_clean":"2024-05-11T12:22:43.844702+0000","last_became_active":"2024-05-11T12:21:08.509498+0000","last_became_peered":"2024-05-11T12:21:08.509498+0000","last_unstale":"2024-05-11T12:22:43.844702+0000","last_undegraded":"2024-05-11T12:22:43.844702+0000","last_fullsized":"2024-05-11T12:22:43.844702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:43.282700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.789945+0000","last_change":"2024-05-11T12:21:11.533108+0000","last_active":"2024-05-11T12:22:43.789945+0000","last_peered":"2024-05-11T12:22:43.789945+0000","last_clean":"2024-05-11T12:22:43.789945+0000","last_became_active":"2024-05-11T12:21:11.532779+0000","last_became_peered":"2024-05-11T12:21:11.532779+0000","last_unstale":"2024-05-11T12:22:43.789945+0000","last_undegraded":"2024-05-11T12:22:43.789945+0000","last_fullsized":"2024-05-11T12:22:43.789945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:55.311786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751266+0000","last_change":"2024-05-11T12:21:08.513355+0000","last_active":"2024-05-11T12:22:43.751266+0000","last_peered":"2024-05-11T12:22:43.751266+0000","last_clean":"2024-05-11T12:22:43.751266+0000","last_became_active":"2024-05-11T12:21:08.512964+0000","last_became_peered":"2024-05-11T12:21:08.512964+0000","last_unstale":"2024-05-11T12:22:43.751266+0000","last_undegraded":"2024-05-11T12:22:43.751266+0000","last_fullsized":"2024-05-11T12:22:43.751266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:12.296865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.646908+0000","last_change":"2024-05-11T12:21:11.525958+0000","last_active":"2024-05-11T12:22:43.646908+0000","last_peered":"2024-05-11T12:22:43.646908+0000","last_clean":"2024-05-11T12:22:43.646908+0000","last_became_active":"2024-05-11T12:21:11.525285+0000","last_became_peered":"2024-05-11T12:21:11.525285+0000","last_unstale":"2024-05-11T12:22:43.646908+0000","last_undegraded":"2024-05-11T12:22:43.646908+0000","last_fullsized":"2024-05-11T12:22:43.646908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:24:47.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.749125+0000","last_change":"2024-05-11T12:21:42.673378+0000","last_active":"2024-05-11T12:22:43.749125+0000","last_peered":"2024-05-11T12:22:43.749125+0000","last_clean":"2024-05-11T12:22:43.749125+0000","last_became_active":"2024-05-11T12:21:42.672870+0000","last_became_peered":"2024-05-11T12:21:42.672870+0000","last_unstale":"2024-05-11T12:22:43.749125+0000","last_undegraded":"2024-05-11T12:22:43.749125+0000","last_fullsized":"2024-05-11T12:22:43.749125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:55:41.845162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.892114+0000","last_change":"2024-05-11T12:21:11.538946+0000","last_active":"2024-05-11T12:22:43.892114+0000","last_peered":"2024-05-11T12:22:43.892114+0000","last_clean":"2024-05-11T12:22:43.892114+0000","last_became_active":"2024-05-11T12:21:11.535804+0000","last_became_peered":"2024-05-11T12:21:11.535804+0000","last_unstale":"2024-05-11T12:22:43.892114+0000","last_undegraded":"2024-05-11T12:22:43.892114+0000","last_fullsized":"2024-05-11T12:22:43.892114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:37:16.221144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649318+0000","last_change":"2024-05-11T12:21:42.674203+0000","last_active":"2024-05-11T12:22:43.649318+0000","last_peered":"2024-05-11T12:22:43.649318+0000","last_clean":"2024-05-11T12:22:43.649318+0000","last_became_active":"2024-05-11T12:21:42.673478+0000","last_became_peered":"2024-05-11T12:21:42.673478+0000","last_unstale":"2024-05-11T12:22:43.649318+0000","last_undegraded":"2024-05-11T12:22:43.649318+0000","last_fullsized":"2024-05-11T12:22:43.649318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:20.648528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650740+0000","last_change":"2024-05-11T12:21:11.535346+0000","last_active":"2024-05-11T12:22:43.650740+0000","last_peered":"2024-05-11T12:22:43.650740+0000","last_clean":"2024-05-11T12:22:43.650740+0000","last_became_active":"2024-05-11T12:21:11.530262+0000","last_became_peered":"2024-05-11T12:21:11.530262+0000","last_unstale":"2024-05-11T12:22:43.650740+0000","last_undegraded":"2024-05-11T12:22:43.650740+0000","last_fullsized":"2024-05-11T12:22:43.650740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:13.169797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.752053+0000","last_change":"2024-05-11T12:21:08.531598+0000","last_active":"2024-05-11T12:22:43.752053+0000","last_peered":"2024-05-11T12:22:43.752053+0000","last_clean":"2024-05-11T12:22:43.752053+0000","last_became_active":"2024-05-11T12:21:08.531158+0000","last_became_peered":"2024-05-11T12:21:08.531158+0000","last_unstale":"2024-05-11T12:22:43.752053+0000","last_undegraded":"2024-05-11T12:22:43.752053+0000","last_fullsized":"2024-05-11T12:22:43.752053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:43.528151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094901+0000","last_change":"2024-05-11T12:22:44.094901+0000","last_active":"2024-05-11T12:22:44.094901+0000","last_peered":"2024-05-11T12:22:44.094901+0000","last_clean":"2024-05-11T12:22:44.094901+0000","last_became_active":"2024-05-11T12:22:44.094563+0000","last_became_peered":"2024-05-11T12:22:44.094563+0000","last_unstale":"2024-05-11T12:22:44.094901+0000","last_undegraded":"2024-05-11T12:22:44.094901+0000","last_fullsized":"2024-05-11T12:22:44.094901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:42:41.499957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,4],"acting":[9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":251,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:58.774180+0000","last_change":"2024-05-11T12:21:08.516737+0000","last_active":"2024-05-11T12:22:58.774180+0000","last_peered":"2024-05-11T12:22:58.774180+0000","last_clean":"2024-05-11T12:22:58.774180+0000","last_became_active":"2024-05-11T12:21:08.516316+0000","last_became_peered":"2024-05-11T12:21:08.516316+0000","last_unstale":"2024-05-11T12:22:58.774180+0000","last_undegraded":"2024-05-11T12:22:58.774180+0000","last_fullsized":"2024-05-11T12:22:58.774180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:25.769713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":193,"num_read_kb":156,"num_write":28,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094134+0000","last_change":"2024-05-11T12:21:11.528553+0000","last_active":"2024-05-11T12:22:44.094134+0000","last_peered":"2024-05-11T12:22:44.094134+0000","last_clean":"2024-05-11T12:22:44.094134+0000","last_became_active":"2024-05-11T12:21:11.528175+0000","last_became_peered":"2024-05-11T12:21:11.528175+0000","last_unstale":"2024-05-11T12:22:44.094134+0000","last_undegraded":"2024-05-11T12:22:44.094134+0000","last_fullsized":"2024-05-11T12:22:44.094134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:46:42.689851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651038+0000","last_change":"2024-05-11T12:21:42.673360+0000","last_active":"2024-05-11T12:22:43.651038+0000","last_peered":"2024-05-11T12:22:43.651038+0000","last_clean":"2024-05-11T12:22:43.651038+0000","last_became_active":"2024-05-11T12:21:42.664842+0000","last_became_peered":"2024-05-11T12:21:42.664842+0000","last_unstale":"2024-05-11T12:22:43.651038+0000","last_undegraded":"2024-05-11T12:22:43.651038+0000","last_fullsized":"2024-05-11T12:22:43.651038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:40.271658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844736+0000","last_change":"2024-05-11T12:21:11.533157+0000","last_active":"2024-05-11T12:22:43.844736+0000","last_peered":"2024-05-11T12:22:43.844736+0000","last_clean":"2024-05-11T12:22:43.844736+0000","last_became_active":"2024-05-11T12:21:11.532814+0000","last_became_peered":"2024-05-11T12:21:11.532814+0000","last_unstale":"2024-05-11T12:22:43.844736+0000","last_undegraded":"2024-05-11T12:22:43.844736+0000","last_fullsized":"2024-05-11T12:22:43.844736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:30:56.555384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751534+0000","last_change":"2024-05-11T12:21:08.511907+0000","last_active":"2024-05-11T12:22:43.751534+0000","last_peered":"2024-05-11T12:22:43.751534+0000","last_clean":"2024-05-11T12:22:43.751534+0000","last_became_active":"2024-05-11T12:21:08.511296+0000","last_became_peered":"2024-05-11T12:21:08.511296+0000","last_unstale":"2024-05-11T12:22:43.751534+0000","last_undegraded":"2024-05-11T12:22:43.751534+0000","last_fullsized":"2024-05-11T12:22:43.751534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:43:17.374216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649287+0000","last_change":"2024-05-11T12:21:11.537859+0000","last_active":"2024-05-11T12:22:43.649287+0000","last_peered":"2024-05-11T12:22:43.649287+0000","last_clean":"2024-05-11T12:22:43.649287+0000","last_became_active":"2024-05-11T12:21:11.537448+0000","last_became_peered":"2024-05-11T12:21:11.537448+0000","last_unstale":"2024-05-11T12:22:43.649287+0000","last_undegraded":"2024-05-11T12:22:43.649287+0000","last_fullsized":"2024-05-11T12:22:43.649287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:09:33.866539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647830+0000","last_change":"2024-05-11T12:21:08.510228+0000","last_active":"2024-05-11T12:22:43.647830+0000","last_peered":"2024-05-11T12:22:43.647830+0000","last_clean":"2024-05-11T12:22:43.647830+0000","last_became_active":"2024-05-11T12:21:08.509671+0000","last_became_peered":"2024-05-11T12:21:08.509671+0000","last_unstale":"2024-05-11T12:22:43.647830+0000","last_undegraded":"2024-05-11T12:22:43.647830+0000","last_fullsized":"2024-05-11T12:22:43.647830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:06.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649052+0000","last_change":"2024-05-11T12:21:11.530935+0000","last_active":"2024-05-11T12:22:43.649052+0000","last_peered":"2024-05-11T12:22:43.649052+0000","last_clean":"2024-05-11T12:22:43.649052+0000","last_became_active":"2024-05-11T12:21:11.530579+0000","last_became_peered":"2024-05-11T12:21:11.530579+0000","last_unstale":"2024-05-11T12:22:43.649052+0000","last_undegraded":"2024-05-11T12:22:43.649052+0000","last_fullsized":"2024-05-11T12:22:43.649052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:38.296021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651027+0000","last_change":"2024-05-11T12:21:42.681114+0000","last_active":"2024-05-11T12:22:43.651027+0000","last_peered":"2024-05-11T12:22:43.651027+0000","last_clean":"2024-05-11T12:22:43.651027+0000","last_became_active":"2024-05-11T12:21:42.666878+0000","last_became_peered":"2024-05-11T12:21:42.666878+0000","last_unstale":"2024-05-11T12:22:43.651027+0000","last_undegraded":"2024-05-11T12:22:43.651027+0000","last_fullsized":"2024-05-11T12:22:43.651027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:45.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.893287+0000","last_change":"2024-05-11T12:22:43.892989+0000","last_active":"2024-05-11T12:22:43.893287+0000","last_peered":"2024-05-11T12:22:43.893287+0000","last_clean":"2024-05-11T12:22:43.893287+0000","last_became_active":"2024-05-11T12:22:43.892599+0000","last_became_peered":"2024-05-11T12:22:43.892599+0000","last_unstale":"2024-05-11T12:22:43.893287+0000","last_undegraded":"2024-05-11T12:22:43.893287+0000","last_fullsized":"2024-05-11T12:22:43.893287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:46.357968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094125+0000","last_change":"2024-05-11T12:21:08.510504+0000","last_active":"2024-05-11T12:22:44.094125+0000","last_peered":"2024-05-11T12:22:44.094125+0000","last_clean":"2024-05-11T12:22:44.094125+0000","last_became_active":"2024-05-11T12:21:08.510074+0000","last_became_peered":"2024-05-11T12:21:08.510074+0000","last_unstale":"2024-05-11T12:22:44.094125+0000","last_undegraded":"2024-05-11T12:22:44.094125+0000","last_fullsized":"2024-05-11T12:22:44.094125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:31:49.465252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.753140+0000","last_change":"2024-05-11T12:22:43.753140+0000","last_active":"2024-05-11T12:22:43.753140+0000","last_peered":"2024-05-11T12:22:43.753140+0000","last_clean":"2024-05-11T12:22:43.753140+0000","last_became_active":"2024-05-11T12:22:43.752693+0000","last_became_peered":"2024-05-11T12:22:43.752693+0000","last_unstale":"2024-05-11T12:22:43.753140+0000","last_undegraded":"2024-05-11T12:22:43.753140+0000","last_fullsized":"2024-05-11T12:22:43.753140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:13.152017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647457+0000","last_change":"2024-05-11T12:21:08.531926+0000","last_active":"2024-05-11T12:22:43.647457+0000","last_peered":"2024-05-11T12:22:43.647457+0000","last_clean":"2024-05-11T12:22:43.647457+0000","last_became_active":"2024-05-11T12:21:08.531548+0000","last_became_peered":"2024-05-11T12:21:08.531548+0000","last_unstale":"2024-05-11T12:22:43.647457+0000","last_undegraded":"2024-05-11T12:22:43.647457+0000","last_fullsized":"2024-05-11T12:22:43.647457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:25:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748981+0000","last_change":"2024-05-11T12:21:11.528595+0000","last_active":"2024-05-11T12:22:43.748981+0000","last_peered":"2024-05-11T12:22:43.748981+0000","last_clean":"2024-05-11T12:22:43.748981+0000","last_became_active":"2024-05-11T12:21:11.527782+0000","last_became_peered":"2024-05-11T12:21:11.527782+0000","last_unstale":"2024-05-11T12:22:43.748981+0000","last_undegraded":"2024-05-11T12:22:43.748981+0000","last_fullsized":"2024-05-11T12:22:43.748981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:31:00.337497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649582+0000","last_change":"2024-05-11T12:21:08.508093+0000","last_active":"2024-05-11T12:22:43.649582+0000","last_peered":"2024-05-11T12:22:43.649582+0000","last_clean":"2024-05-11T12:22:43.649582+0000","last_became_active":"2024-05-11T12:21:08.507710+0000","last_became_peered":"2024-05-11T12:21:08.507710+0000","last_unstale":"2024-05-11T12:22:43.649582+0000","last_undegraded":"2024-05-11T12:22:43.649582+0000","last_fullsized":"2024-05-11T12:22:43.649582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:24.273932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748841+0000","last_change":"2024-05-11T12:21:11.526026+0000","last_active":"2024-05-11T12:22:43.748841+0000","last_peered":"2024-05-11T12:22:43.748841+0000","last_clean":"2024-05-11T12:22:43.748841+0000","last_became_active":"2024-05-11T12:21:11.525628+0000","last_became_peered":"2024-05-11T12:21:11.525628+0000","last_unstale":"2024-05-11T12:22:43.748841+0000","last_undegraded":"2024-05-11T12:22:43.748841+0000","last_fullsized":"2024-05-11T12:22:43.748841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:12.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751702+0000","last_change":"2024-05-11T12:21:11.535808+0000","last_active":"2024-05-11T12:22:43.751702+0000","last_peered":"2024-05-11T12:22:43.751702+0000","last_clean":"2024-05-11T12:22:43.751702+0000","last_became_active":"2024-05-11T12:21:11.535204+0000","last_became_peered":"2024-05-11T12:21:11.535204+0000","last_unstale":"2024-05-11T12:22:43.751702+0000","last_undegraded":"2024-05-11T12:22:43.751702+0000","last_fullsized":"2024-05-11T12:22:43.751702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:36:01.208543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844596+0000","last_change":"2024-05-11T12:21:08.531838+0000","last_active":"2024-05-11T12:22:43.844596+0000","last_peered":"2024-05-11T12:22:43.844596+0000","last_clean":"2024-05-11T12:22:43.844596+0000","last_became_active":"2024-05-11T12:21:08.531491+0000","last_became_peered":"2024-05-11T12:21:08.531491+0000","last_unstale":"2024-05-11T12:22:43.844596+0000","last_undegraded":"2024-05-11T12:22:43.844596+0000","last_fullsized":"2024-05-11T12:22:43.844596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:33.928985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790145+0000","last_change":"2024-05-11T12:21:11.538563+0000","last_active":"2024-05-11T12:22:43.790145+0000","last_peered":"2024-05-11T12:22:43.790145+0000","last_clean":"2024-05-11T12:22:43.790145+0000","last_became_active":"2024-05-11T12:21:11.538200+0000","last_became_peered":"2024-05-11T12:21:11.538200+0000","last_unstale":"2024-05-11T12:22:43.790145+0000","last_undegraded":"2024-05-11T12:22:43.790145+0000","last_fullsized":"2024-05-11T12:22:43.790145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:18.879528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648338+0000","last_change":"2024-05-11T12:21:08.509972+0000","last_active":"2024-05-11T12:22:43.648338+0000","last_peered":"2024-05-11T12:22:43.648338+0000","last_clean":"2024-05-11T12:22:43.648338+0000","last_became_active":"2024-05-11T12:21:08.509531+0000","last_became_peered":"2024-05-11T12:21:08.509531+0000","last_unstale":"2024-05-11T12:22:43.648338+0000","last_undegraded":"2024-05-11T12:22:43.648338+0000","last_fullsized":"2024-05-11T12:22:43.648338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:47:13.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649321+0000","last_change":"2024-05-11T12:19:42.546860+0000","last_active":"2024-05-11T12:22:43.649321+0000","last_peered":"2024-05-11T12:22:43.649321+0000","last_clean":"2024-05-11T12:22:43.649321+0000","last_became_active":"2024-05-11T12:19:42.541449+0000","last_became_peered":"2024-05-11T12:19:42.541449+0000","last_unstale":"2024-05-11T12:22:43.649321+0000","last_undegraded":"2024-05-11T12:22:43.649321+0000","last_fullsized":"2024-05-11T12:22:43.649321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_clean_scrub_stamp":"2024-05-11T12:16:12.381187+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:45.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844576+0000","last_change":"2024-05-11T12:21:11.536770+0000","last_active":"2024-05-11T12:22:43.844576+0000","last_peered":"2024-05-11T12:22:43.844576+0000","last_clean":"2024-05-11T12:22:43.844576+0000","last_became_active":"2024-05-11T12:21:11.536421+0000","last_became_peered":"2024-05-11T12:21:11.536421+0000","last_unstale":"2024-05-11T12:22:43.844576+0000","last_undegraded":"2024-05-11T12:22:43.844576+0000","last_fullsized":"2024-05-11T12:22:43.844576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:42.217398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649287+0000","last_change":"2024-05-11T12:21:42.675283+0000","last_active":"2024-05-11T12:22:43.649287+0000","last_peered":"2024-05-11T12:22:43.649287+0000","last_clean":"2024-05-11T12:22:43.649287+0000","last_became_active":"2024-05-11T12:21:42.674775+0000","last_became_peered":"2024-05-11T12:21:42.674775+0000","last_unstale":"2024-05-11T12:22:43.649287+0000","last_undegraded":"2024-05-11T12:22:43.649287+0000","last_fullsized":"2024-05-11T12:22:43.649287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:13:30.143988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094167+0000","last_change":"2024-05-11T12:21:08.533462+0000","last_active":"2024-05-11T12:22:44.094167+0000","last_peered":"2024-05-11T12:22:44.094167+0000","last_clean":"2024-05-11T12:22:44.094167+0000","last_became_active":"2024-05-11T12:21:08.533081+0000","last_became_peered":"2024-05-11T12:21:08.533081+0000","last_unstale":"2024-05-11T12:22:44.094167+0000","last_undegraded":"2024-05-11T12:22:44.094167+0000","last_fullsized":"2024-05-11T12:22:44.094167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:01:48.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844875+0000","last_change":"2024-05-11T12:21:11.539484+0000","last_active":"2024-05-11T12:22:43.844875+0000","last_peered":"2024-05-11T12:22:43.844875+0000","last_clean":"2024-05-11T12:22:43.844875+0000","last_became_active":"2024-05-11T12:21:11.536810+0000","last_became_peered":"2024-05-11T12:21:11.536810+0000","last_unstale":"2024-05-11T12:22:43.844875+0000","last_undegraded":"2024-05-11T12:22:43.844875+0000","last_fullsized":"2024-05-11T12:22:43.844875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:47:41.461766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647372+0000","last_change":"2024-05-11T12:21:08.510519+0000","last_active":"2024-05-11T12:22:43.647372+0000","last_peered":"2024-05-11T12:22:43.647372+0000","last_clean":"2024-05-11T12:22:43.647372+0000","last_became_active":"2024-05-11T12:21:08.510180+0000","last_became_peered":"2024-05-11T12:21:08.510180+0000","last_unstale":"2024-05-11T12:22:43.647372+0000","last_undegraded":"2024-05-11T12:22:43.647372+0000","last_fullsized":"2024-05-11T12:22:43.647372+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:24.537043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790387+0000","last_change":"2024-05-11T12:21:11.533726+0000","last_active":"2024-05-11T12:22:43.790387+0000","last_peered":"2024-05-11T12:22:43.790387+0000","last_clean":"2024-05-11T12:22:43.790387+0000","last_became_active":"2024-05-11T12:21:11.533393+0000","last_became_peered":"2024-05-11T12:21:11.533393+0000","last_unstale":"2024-05-11T12:22:43.790387+0000","last_undegraded":"2024-05-11T12:22:43.790387+0000","last_fullsized":"2024-05-11T12:22:43.790387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:03.071666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648325+0000","last_change":"2024-05-11T12:21:09.036622+0000","last_active":"2024-05-11T12:22:43.648325+0000","last_peered":"2024-05-11T12:22:43.648325+0000","last_clean":"2024-05-11T12:22:43.648325+0000","last_became_active":"2024-05-11T12:21:09.036209+0000","last_became_peered":"2024-05-11T12:21:09.036209+0000","last_unstale":"2024-05-11T12:22:43.648325+0000","last_undegraded":"2024-05-11T12:22:43.648325+0000","last_fullsized":"2024-05-11T12:22:43.648325+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:29:58.947588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.846111+0000","last_change":"2024-05-11T12:22:43.846111+0000","last_active":"2024-05-11T12:22:43.846111+0000","last_peered":"2024-05-11T12:22:43.846111+0000","last_clean":"2024-05-11T12:22:43.846111+0000","last_became_active":"2024-05-11T12:22:43.845712+0000","last_became_peered":"2024-05-11T12:22:43.845712+0000","last_unstale":"2024-05-11T12:22:43.846111+0000","last_undegraded":"2024-05-11T12:22:43.846111+0000","last_fullsized":"2024-05-11T12:22:43.846111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:19:47.516917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891550+0000","last_change":"2024-05-11T12:21:08.533435+0000","last_active":"2024-05-11T12:22:43.891550+0000","last_peered":"2024-05-11T12:22:43.891550+0000","last_clean":"2024-05-11T12:22:43.891550+0000","last_became_active":"2024-05-11T12:21:08.533100+0000","last_became_peered":"2024-05-11T12:21:08.533100+0000","last_unstale":"2024-05-11T12:22:43.891550+0000","last_undegraded":"2024-05-11T12:22:43.891550+0000","last_fullsized":"2024-05-11T12:22:43.891550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:17.619015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093615+0000","last_change":"2024-05-11T12:21:11.526140+0000","last_active":"2024-05-11T12:22:44.093615+0000","last_peered":"2024-05-11T12:22:44.093615+0000","last_clean":"2024-05-11T12:22:44.093615+0000","last_became_active":"2024-05-11T12:21:11.525810+0000","last_became_peered":"2024-05-11T12:21:11.525810+0000","last_unstale":"2024-05-11T12:22:44.093615+0000","last_undegraded":"2024-05-11T12:22:44.093615+0000","last_fullsized":"2024-05-11T12:22:44.093615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:14:04.160379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790069+0000","last_change":"2024-05-11T12:21:08.508031+0000","last_active":"2024-05-11T12:22:43.790069+0000","last_peered":"2024-05-11T12:22:43.790069+0000","last_clean":"2024-05-11T12:22:43.790069+0000","last_became_active":"2024-05-11T12:21:08.507703+0000","last_became_peered":"2024-05-11T12:21:08.507703+0000","last_unstale":"2024-05-11T12:22:43.790069+0000","last_undegraded":"2024-05-11T12:22:43.790069+0000","last_fullsized":"2024-05-11T12:22:43.790069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:05.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651158+0000","last_change":"2024-05-11T12:21:11.528293+0000","last_active":"2024-05-11T12:22:43.651158+0000","last_peered":"2024-05-11T12:22:43.651158+0000","last_clean":"2024-05-11T12:22:43.651158+0000","last_became_active":"2024-05-11T12:21:11.527862+0000","last_became_peered":"2024-05-11T12:21:11.527862+0000","last_unstale":"2024-05-11T12:22:43.651158+0000","last_undegraded":"2024-05-11T12:22:43.651158+0000","last_fullsized":"2024-05-11T12:22:43.651158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:07.335829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653576+0000","last_change":"2024-05-11T12:21:08.510138+0000","last_active":"2024-05-11T12:22:43.653576+0000","last_peered":"2024-05-11T12:22:43.653576+0000","last_clean":"2024-05-11T12:22:43.653576+0000","last_became_active":"2024-05-11T12:21:08.509566+0000","last_became_peered":"2024-05-11T12:21:08.509566+0000","last_unstale":"2024-05-11T12:22:43.653576+0000","last_undegraded":"2024-05-11T12:22:43.653576+0000","last_fullsized":"2024-05-11T12:22:43.653576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:24:58.174305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.750630+0000","last_change":"2024-05-11T12:22:43.750230+0000","last_active":"2024-05-11T12:22:43.750630+0000","last_peered":"2024-05-11T12:22:43.750630+0000","last_clean":"2024-05-11T12:22:43.750630+0000","last_became_active":"2024-05-11T12:22:43.749836+0000","last_became_peered":"2024-05-11T12:22:43.749836+0000","last_unstale":"2024-05-11T12:22:43.750630+0000","last_undegraded":"2024-05-11T12:22:43.750630+0000","last_fullsized":"2024-05-11T12:22:43.750630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:36.498862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093884+0000","last_change":"2024-05-11T12:21:42.683706+0000","last_active":"2024-05-11T12:22:44.093884+0000","last_peered":"2024-05-11T12:22:44.093884+0000","last_clean":"2024-05-11T12:22:44.093884+0000","last_became_active":"2024-05-11T12:21:42.675791+0000","last_became_peered":"2024-05-11T12:21:42.675791+0000","last_unstale":"2024-05-11T12:22:44.093884+0000","last_undegraded":"2024-05-11T12:22:44.093884+0000","last_fullsized":"2024-05-11T12:22:44.093884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:13:26.737525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4],"acting":[8,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649661+0000","last_change":"2024-05-11T12:21:11.544153+0000","last_active":"2024-05-11T12:22:43.649661+0000","last_peered":"2024-05-11T12:22:43.649661+0000","last_clean":"2024-05-11T12:22:43.649661+0000","last_became_active":"2024-05-11T12:21:11.543772+0000","last_became_peered":"2024-05-11T12:21:11.543772+0000","last_unstale":"2024-05-11T12:22:43.649661+0000","last_undegraded":"2024-05-11T12:22:43.649661+0000","last_fullsized":"2024-05-11T12:22:43.649661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:54:30.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653816+0000","last_change":"2024-05-11T12:21:09.036409+0000","last_active":"2024-05-11T12:22:43.653816+0000","last_peered":"2024-05-11T12:22:43.653816+0000","last_clean":"2024-05-11T12:22:43.653816+0000","last_became_active":"2024-05-11T12:21:09.036009+0000","last_became_peered":"2024-05-11T12:21:09.036009+0000","last_unstale":"2024-05-11T12:22:43.653816+0000","last_undegraded":"2024-05-11T12:22:43.653816+0000","last_fullsized":"2024-05-11T12:22:43.653816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:55:28.736718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649738+0000","last_change":"2024-05-11T12:21:11.542685+0000","last_active":"2024-05-11T12:22:43.649738+0000","last_peered":"2024-05-11T12:22:43.649738+0000","last_clean":"2024-05-11T12:22:43.649738+0000","last_became_active":"2024-05-11T12:21:11.542130+0000","last_became_peered":"2024-05-11T12:21:11.542130+0000","last_unstale":"2024-05-11T12:22:43.649738+0000","last_undegraded":"2024-05-11T12:22:43.649738+0000","last_fullsized":"2024-05-11T12:22:43.649738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:22.578809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751889+0000","last_change":"2024-05-11T12:21:08.510869+0000","last_active":"2024-05-11T12:22:43.751889+0000","last_peered":"2024-05-11T12:22:43.751889+0000","last_clean":"2024-05-11T12:22:43.751889+0000","last_became_active":"2024-05-11T12:21:08.510510+0000","last_became_peered":"2024-05-11T12:21:08.510510+0000","last_unstale":"2024-05-11T12:22:43.751889+0000","last_undegraded":"2024-05-11T12:22:43.751889+0000","last_fullsized":"2024-05-11T12:22:43.751889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:50.002313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649208+0000","last_change":"2024-05-11T12:21:11.538622+0000","last_active":"2024-05-11T12:22:43.649208+0000","last_peered":"2024-05-11T12:22:43.649208+0000","last_clean":"2024-05-11T12:22:43.649208+0000","last_became_active":"2024-05-11T12:21:11.538229+0000","last_became_peered":"2024-05-11T12:21:11.538229+0000","last_unstale":"2024-05-11T12:22:43.649208+0000","last_undegraded":"2024-05-11T12:22:43.649208+0000","last_fullsized":"2024-05-11T12:22:43.649208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:01:15.912428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.845041+0000","last_change":"2024-05-11T12:21:42.677421+0000","last_active":"2024-05-11T12:22:43.845041+0000","last_peered":"2024-05-11T12:22:43.845041+0000","last_clean":"2024-05-11T12:22:43.845041+0000","last_became_active":"2024-05-11T12:21:42.672695+0000","last_became_peered":"2024-05-11T12:21:42.672695+0000","last_unstale":"2024-05-11T12:22:43.845041+0000","last_undegraded":"2024-05-11T12:22:43.845041+0000","last_fullsized":"2024-05-11T12:22:43.845041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:30:27.606816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649514+0000","last_change":"2024-05-11T12:21:11.536149+0000","last_active":"2024-05-11T12:22:43.649514+0000","last_peered":"2024-05-11T12:22:43.649514+0000","last_clean":"2024-05-11T12:22:43.649514+0000","last_became_active":"2024-05-11T12:21:11.535763+0000","last_became_peered":"2024-05-11T12:21:11.535763+0000","last_unstale":"2024-05-11T12:22:43.649514+0000","last_undegraded":"2024-05-11T12:22:43.649514+0000","last_fullsized":"2024-05-11T12:22:43.649514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:46:15.246610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651216+0000","last_change":"2024-05-11T12:21:11.534318+0000","last_active":"2024-05-11T12:22:43.651216+0000","last_peered":"2024-05-11T12:22:43.651216+0000","last_clean":"2024-05-11T12:22:43.651216+0000","last_became_active":"2024-05-11T12:21:11.533949+0000","last_became_peered":"2024-05-11T12:21:11.533949+0000","last_unstale":"2024-05-11T12:22:43.651216+0000","last_undegraded":"2024-05-11T12:22:43.651216+0000","last_fullsized":"2024-05-11T12:22:43.651216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:41:42.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653528+0000","last_change":"2024-05-11T12:21:11.529276+0000","last_active":"2024-05-11T12:22:43.653528+0000","last_peered":"2024-05-11T12:22:43.653528+0000","last_clean":"2024-05-11T12:22:43.653528+0000","last_became_active":"2024-05-11T12:21:11.528857+0000","last_became_peered":"2024-05-11T12:21:11.528857+0000","last_unstale":"2024-05-11T12:22:43.653528+0000","last_undegraded":"2024-05-11T12:22:43.653528+0000","last_fullsized":"2024-05-11T12:22:43.653528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:54.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653362+0000","last_change":"2024-05-11T12:21:11.532342+0000","last_active":"2024-05-11T12:22:43.653362+0000","last_peered":"2024-05-11T12:22:43.653362+0000","last_clean":"2024-05-11T12:22:43.653362+0000","last_became_active":"2024-05-11T12:21:11.531949+0000","last_became_peered":"2024-05-11T12:21:11.531949+0000","last_unstale":"2024-05-11T12:22:43.653362+0000","last_undegraded":"2024-05-11T12:22:43.653362+0000","last_fullsized":"2024-05-11T12:22:43.653362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:23:00.010874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650629+0000","last_change":"2024-05-11T12:21:11.531547+0000","last_active":"2024-05-11T12:22:43.650629+0000","last_peered":"2024-05-11T12:22:43.650629+0000","last_clean":"2024-05-11T12:22:43.650629+0000","last_became_active":"2024-05-11T12:21:11.529702+0000","last_became_peered":"2024-05-11T12:21:11.529702+0000","last_unstale":"2024-05-11T12:22:43.650629+0000","last_undegraded":"2024-05-11T12:22:43.650629+0000","last_fullsized":"2024-05-11T12:22:43.650629+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:13.857026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844937+0000","last_change":"2024-05-11T12:21:11.531261+0000","last_active":"2024-05-11T12:22:43.844937+0000","last_peered":"2024-05-11T12:22:43.844937+0000","last_clean":"2024-05-11T12:22:43.844937+0000","last_became_active":"2024-05-11T12:21:11.530561+0000","last_became_peered":"2024-05-11T12:21:11.530561+0000","last_unstale":"2024-05-11T12:22:43.844937+0000","last_undegraded":"2024-05-11T12:22:43.844937+0000","last_fullsized":"2024-05-11T12:22:43.844937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:52:07.654417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651599+0000","last_change":"2024-05-11T12:21:11.533255+0000","last_active":"2024-05-11T12:22:43.651599+0000","last_peered":"2024-05-11T12:22:43.651599+0000","last_clean":"2024-05-11T12:22:43.651599+0000","last_became_active":"2024-05-11T12:21:11.532909+0000","last_became_peered":"2024-05-11T12:21:11.532909+0000","last_unstale":"2024-05-11T12:22:43.651599+0000","last_undegraded":"2024-05-11T12:22:43.651599+0000","last_fullsized":"2024-05-11T12:22:43.651599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:07:39.501482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653835+0000","last_change":"2024-05-11T12:21:11.528856+0000","last_active":"2024-05-11T12:22:43.653835+0000","last_peered":"2024-05-11T12:22:43.653835+0000","last_clean":"2024-05-11T12:22:43.653835+0000","last_became_active":"2024-05-11T12:21:11.528230+0000","last_became_peered":"2024-05-11T12:21:11.528230+0000","last_unstale":"2024-05-11T12:22:43.653835+0000","last_undegraded":"2024-05-11T12:22:43.653835+0000","last_fullsized":"2024-05-11T12:22:43.653835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:40.429045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790285+0000","last_change":"2024-05-11T12:21:11.534606+0000","last_active":"2024-05-11T12:22:43.790285+0000","last_peered":"2024-05-11T12:22:43.790285+0000","last_clean":"2024-05-11T12:22:43.790285+0000","last_became_active":"2024-05-11T12:21:11.534193+0000","last_became_peered":"2024-05-11T12:21:11.534193+0000","last_unstale":"2024-05-11T12:22:43.790285+0000","last_undegraded":"2024-05-11T12:22:43.790285+0000","last_fullsized":"2024-05-11T12:22:43.790285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:00:12.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751347+0000","last_change":"2024-05-11T12:21:11.525959+0000","last_active":"2024-05-11T12:22:43.751347+0000","last_peered":"2024-05-11T12:22:43.751347+0000","last_clean":"2024-05-11T12:22:43.751347+0000","last_became_active":"2024-05-11T12:21:11.525129+0000","last_became_peered":"2024-05-11T12:21:11.525129+0000","last_unstale":"2024-05-11T12:22:43.751347+0000","last_undegraded":"2024-05-11T12:22:43.751347+0000","last_fullsized":"2024-05-11T12:22:43.751347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:28.828336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790042+0000","last_change":"2024-05-11T12:21:11.538591+0000","last_active":"2024-05-11T12:22:43.790042+0000","last_peered":"2024-05-11T12:22:43.790042+0000","last_clean":"2024-05-11T12:22:43.790042+0000","last_became_active":"2024-05-11T12:21:11.537855+0000","last_became_peered":"2024-05-11T12:21:11.537855+0000","last_unstale":"2024-05-11T12:22:43.790042+0000","last_undegraded":"2024-05-11T12:22:43.790042+0000","last_fullsized":"2024-05-11T12:22:43.790042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:52.625763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891800+0000","last_change":"2024-05-11T12:21:11.541185+0000","last_active":"2024-05-11T12:22:43.891800+0000","last_peered":"2024-05-11T12:22:43.891800+0000","last_clean":"2024-05-11T12:22:43.891800+0000","last_became_active":"2024-05-11T12:21:11.540799+0000","last_became_peered":"2024-05-11T12:21:11.540799+0000","last_unstale":"2024-05-11T12:22:43.891800+0000","last_undegraded":"2024-05-11T12:22:43.891800+0000","last_fullsized":"2024-05-11T12:22:43.891800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:24.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844818+0000","last_change":"2024-05-11T12:21:11.527811+0000","last_active":"2024-05-11T12:22:43.844818+0000","last_peered":"2024-05-11T12:22:43.844818+0000","last_clean":"2024-05-11T12:22:43.844818+0000","last_became_active":"2024-05-11T12:21:11.526603+0000","last_became_peered":"2024-05-11T12:21:11.526603+0000","last_unstale":"2024-05-11T12:22:43.844818+0000","last_undegraded":"2024-05-11T12:22:43.844818+0000","last_fullsized":"2024-05-11T12:22:43.844818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:43.833506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651229+0000","last_change":"2024-05-11T12:21:11.530824+0000","last_active":"2024-05-11T12:22:43.651229+0000","last_peered":"2024-05-11T12:22:43.651229+0000","last_clean":"2024-05-11T12:22:43.651229+0000","last_became_active":"2024-05-11T12:21:11.530486+0000","last_became_peered":"2024-05-11T12:21:11.530486+0000","last_unstale":"2024-05-11T12:22:43.651229+0000","last_undegraded":"2024-05-11T12:22:43.651229+0000","last_fullsized":"2024-05-11T12:22:43.651229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:47.054822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247249,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62548,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":199,"num_read_kb":162,"num_write":85,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":187644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678055,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":445299,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874283,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":1097694,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037809,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":445847,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234038,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":445869,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430268,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":1036332,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593793,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":445382,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":507015,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":445355,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":507015,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149777,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":445603,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953548,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":445287,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":1035932,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:23:06.090 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph pg dump --format=json 2024-05-11T12:23:06.345 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:06.762 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:06 smithi047 ceph-mon[30375]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:23:06.762 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:06 smithi047 ceph-mon[30375]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:06.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:06 smithi050 ceph-mon[34072]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:23:06.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:06 smithi050 ceph-mon[34072]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:06 smithi064 ceph-mon[34537]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:23:07.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:06 smithi064 ceph-mon[34537]: pgmap v355: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 -- 172.21.15.47:0/1075868964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06c640 msgr2=0x7f624c1103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 --2- 172.21.15.47:0/1075868964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06c640 0x7f624c1103e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f624400b0a0 tx=0x7f624400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 -- 172.21.15.47:0/1075868964 shutdown_connections 2024-05-11T12:23:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 --2- 172.21.15.47:0/1075868964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06c640 0x7f624c1103e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 --2- 172.21.15.47:0/1075868964 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f624c1095c0 0x7f624c1099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 --2- 172.21.15.47:0/1075868964 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f624c06bc80 0x7f624c06c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 -- 172.21.15.47:0/1075868964 >> 172.21.15.47:0/1075868964 conn(0x7f624c079400 msgr2=0x7f624c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 -- 172.21.15.47:0/1075868964 shutdown_connections 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 -- 172.21.15.47:0/1075868964 wait complete. 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 Processor -- start 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 -- start start 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06bc80 0x7f624c19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.167+0000 7f625318f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f624c06c640 0x7f624c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f625318f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f624c1095c0 0x7f624c199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f625318f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f624c1136d0 con 0x7f624c06bc80 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f625318f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f624c113550 con 0x7f624c06c640 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f625318f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f624c113850 con 0x7f624c1095c0 2024-05-11T12:23:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06bc80 0x7f624c19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06bc80 0x7f624c19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46418/0 (socket says 172.21.15.47:46418) 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 -- 172.21.15.47:0/1346050028 learned_addr learned my addr 172.21.15.47:0/1346050028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f625172c700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f624c1095c0 0x7f624c199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f624bfff700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f624c06c640 0x7f624c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 -- 172.21.15.47:0/1346050028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f624c1095c0 msgr2=0x7f624c199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f624c1095c0 0x7f624c199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 -- 172.21.15.47:0/1346050028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f624c06c640 msgr2=0x7f624c19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f624c06c640 0x7f624c19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 -- 172.21.15.47:0/1346050028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c19a0f0 con 0x7f624c06bc80 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f624bfff700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f624c06c640 0x7f624c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6250f2b700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06bc80 0x7f624c19f780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f624000c9b0 tx=0x7f624000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6249ffb700 1 -- 172.21.15.47:0/1346050028 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240007b00 con 0x7f624c06bc80 2024-05-11T12:23:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6249ffb700 1 -- 172.21.15.47:0/1346050028 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6240007ca0 con 0x7f624c06bc80 2024-05-11T12:23:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c19a3e0 con 0x7f624c06bc80 2024-05-11T12:23:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.168+0000 7f6249ffb700 1 -- 172.21.15.47:0/1346050028 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240018660 con 0x7f624c06bc80 2024-05-11T12:23:08.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.169+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c1a6650 con 0x7f624c06bc80 2024-05-11T12:23:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.170+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624c06a240 con 0x7f624c06bc80 2024-05-11T12:23:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.171+0000 7f6249ffb700 1 -- 172.21.15.47:0/1346050028 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f624000f450 con 0x7f624c06bc80 2024-05-11T12:23:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.172+0000 7f6249ffb700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f623c0b2910 0x7f623c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.172+0000 7f624bfff700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f623c0b2910 0x7f623c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.172+0000 7f6249ffb700 1 -- 172.21.15.47:0/1346050028 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f62400e26e0 con 0x7f624c06bc80 2024-05-11T12:23:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.172+0000 7f624bfff700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f623c0b2910 0x7f623c0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f624c079a00 tx=0x7f6238005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.175+0000 7f6249ffb700 1 -- 172.21.15.47:0/1346050028 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6240014030 con 0x7f624c06bc80 2024-05-11T12:23:08.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.388+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f624c063990 con 0x7f623c0b2910 2024-05-11T12:23:08.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.395+0000 7f6249ffb700 1 -- 172.21.15.47:0/1346050028 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258890 (secure 0 0 0) 0x7f624c063990 con 0x7f623c0b2910 2024-05-11T12:23:08.398 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f623c0b2910 msgr2=0x7f623c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f623c0b2910 0x7f623c0b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f624c079a00 tx=0x7f6238005f50 comp rx=0 tx=0).stop 2024-05-11T12:23:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06bc80 msgr2=0x7f624c19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06bc80 0x7f624c19f780 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f624000c9b0 tx=0x7f624000ccc0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 shutdown_connections 2024-05-11T12:23:08.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f624c1095c0 0x7f624c199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624c06bc80 0x7f624c19f780 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f623c0b2910 0x7f623c0b4dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 --2- 172.21.15.47:0/1346050028 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f624c06c640 0x7f624c19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 >> 172.21.15.47:0/1346050028 conn(0x7f624c079400 msgr2=0x7f624c101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 shutdown_connections 2024-05-11T12:23:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:08.398+0000 7f625318f700 1 -- 172.21.15.47:0/1346050028 wait complete. 2024-05-11T12:23:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-05-11T12:23:08.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:08 smithi050 ceph-mon[34072]: pgmap v356: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:09.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:08 smithi047 ceph-mon[30375]: pgmap v356: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:08 smithi064 ceph-mon[34537]: pgmap v356: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:09.142 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-05-11T12:23:07.560261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653150,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":285,"num_read_kb":231,"num_write":210,"num_write_kb":2239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951874,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164,"ondisk_log_size":164,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393400,"kb_used_data":10896,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124548424,"statfs":{"total":1151940427776,"available":1151537586176,"internally_reserved":0,"allocated":11157504,"data_stored":7302630,"data_compressed":246459,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003100"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651849+0000","last_change":"2024-05-11T12:21:11.536657+0000","last_active":"2024-05-11T12:22:43.651849+0000","last_peered":"2024-05-11T12:22:43.651849+0000","last_clean":"2024-05-11T12:22:43.651849+0000","last_became_active":"2024-05-11T12:21:11.536243+0000","last_became_peered":"2024-05-11T12:21:11.536243+0000","last_unstale":"2024-05-11T12:22:43.651849+0000","last_undegraded":"2024-05-11T12:22:43.651849+0000","last_fullsized":"2024-05-11T12:22:43.651849+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:29:20.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650830+0000","last_change":"2024-05-11T12:21:11.527602+0000","last_active":"2024-05-11T12:22:43.650830+0000","last_peered":"2024-05-11T12:22:43.650830+0000","last_clean":"2024-05-11T12:22:43.650830+0000","last_became_active":"2024-05-11T12:21:11.527309+0000","last_became_peered":"2024-05-11T12:21:11.527309+0000","last_unstale":"2024-05-11T12:22:43.650830+0000","last_undegraded":"2024-05-11T12:22:43.650830+0000","last_fullsized":"2024-05-11T12:22:43.650830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:03.764570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650532+0000","last_change":"2024-05-11T12:21:11.529094+0000","last_active":"2024-05-11T12:22:43.650532+0000","last_peered":"2024-05-11T12:22:43.650532+0000","last_clean":"2024-05-11T12:22:43.650532+0000","last_became_active":"2024-05-11T12:21:11.528796+0000","last_became_peered":"2024-05-11T12:21:11.528796+0000","last_unstale":"2024-05-11T12:22:43.650532+0000","last_undegraded":"2024-05-11T12:22:43.650532+0000","last_fullsized":"2024-05-11T12:22:43.650532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:46:41.098066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648539+0000","last_change":"2024-05-11T12:21:11.528903+0000","last_active":"2024-05-11T12:22:43.648539+0000","last_peered":"2024-05-11T12:22:43.648539+0000","last_clean":"2024-05-11T12:22:43.648539+0000","last_became_active":"2024-05-11T12:21:11.528545+0000","last_became_peered":"2024-05-11T12:21:11.528545+0000","last_unstale":"2024-05-11T12:22:43.648539+0000","last_undegraded":"2024-05-11T12:22:43.648539+0000","last_fullsized":"2024-05-11T12:22:43.648539+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:56.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093747+0000","last_change":"2024-05-11T12:21:11.521074+0000","last_active":"2024-05-11T12:22:44.093747+0000","last_peered":"2024-05-11T12:22:44.093747+0000","last_clean":"2024-05-11T12:22:44.093747+0000","last_became_active":"2024-05-11T12:21:11.520675+0000","last_became_peered":"2024-05-11T12:21:11.520675+0000","last_unstale":"2024-05-11T12:22:44.093747+0000","last_undegraded":"2024-05-11T12:22:44.093747+0000","last_fullsized":"2024-05-11T12:22:44.093747+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:22:50.952687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748814+0000","last_change":"2024-05-11T12:21:42.671243+0000","last_active":"2024-05-11T12:22:43.748814+0000","last_peered":"2024-05-11T12:22:43.748814+0000","last_clean":"2024-05-11T12:22:43.748814+0000","last_became_active":"2024-05-11T12:21:42.670801+0000","last_became_peered":"2024-05-11T12:21:42.670801+0000","last_unstale":"2024-05-11T12:22:43.748814+0000","last_undegraded":"2024-05-11T12:22:43.748814+0000","last_fullsized":"2024-05-11T12:22:43.748814+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:02.539855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.095706+0000","last_change":"2024-05-11T12:22:44.095706+0000","last_active":"2024-05-11T12:22:44.095706+0000","last_peered":"2024-05-11T12:22:44.095706+0000","last_clean":"2024-05-11T12:22:44.095706+0000","last_became_active":"2024-05-11T12:22:44.095177+0000","last_became_peered":"2024-05-11T12:22:44.095177+0000","last_unstale":"2024-05-11T12:22:44.095706+0000","last_undegraded":"2024-05-11T12:22:44.095706+0000","last_fullsized":"2024-05-11T12:22:44.095706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:11.426607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.789981+0000","last_change":"2024-05-11T12:21:11.532874+0000","last_active":"2024-05-11T12:22:43.789981+0000","last_peered":"2024-05-11T12:22:43.789981+0000","last_clean":"2024-05-11T12:22:43.789981+0000","last_became_active":"2024-05-11T12:21:11.532237+0000","last_became_peered":"2024-05-11T12:21:11.532237+0000","last_unstale":"2024-05-11T12:22:43.789981+0000","last_undegraded":"2024-05-11T12:22:43.789981+0000","last_fullsized":"2024-05-11T12:22:43.789981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:13.770719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.892174+0000","last_change":"2024-05-11T12:21:11.536465+0000","last_active":"2024-05-11T12:22:43.892174+0000","last_peered":"2024-05-11T12:22:43.892174+0000","last_clean":"2024-05-11T12:22:43.892174+0000","last_became_active":"2024-05-11T12:21:11.536057+0000","last_became_peered":"2024-05-11T12:21:11.536057+0000","last_unstale":"2024-05-11T12:22:43.892174+0000","last_undegraded":"2024-05-11T12:22:43.892174+0000","last_fullsized":"2024-05-11T12:22:43.892174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:22:45.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.749219+0000","last_change":"2024-05-11T12:21:11.528187+0000","last_active":"2024-05-11T12:22:43.749219+0000","last_peered":"2024-05-11T12:22:43.749219+0000","last_clean":"2024-05-11T12:22:43.749219+0000","last_became_active":"2024-05-11T12:21:11.527780+0000","last_became_peered":"2024-05-11T12:21:11.527780+0000","last_unstale":"2024-05-11T12:22:43.749219+0000","last_undegraded":"2024-05-11T12:22:43.749219+0000","last_fullsized":"2024-05-11T12:22:43.749219+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:23.484372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649613+0000","last_change":"2024-05-11T12:21:11.543388+0000","last_active":"2024-05-11T12:22:43.649613+0000","last_peered":"2024-05-11T12:22:43.649613+0000","last_clean":"2024-05-11T12:22:43.649613+0000","last_became_active":"2024-05-11T12:21:11.542984+0000","last_became_peered":"2024-05-11T12:21:11.542984+0000","last_unstale":"2024-05-11T12:22:43.649613+0000","last_undegraded":"2024-05-11T12:22:43.649613+0000","last_fullsized":"2024-05-11T12:22:43.649613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:52.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648390+0000","last_change":"2024-05-11T12:21:11.523355+0000","last_active":"2024-05-11T12:22:43.648390+0000","last_peered":"2024-05-11T12:22:43.648390+0000","last_clean":"2024-05-11T12:22:43.648390+0000","last_became_active":"2024-05-11T12:21:11.523007+0000","last_became_peered":"2024-05-11T12:21:11.523007+0000","last_unstale":"2024-05-11T12:22:43.648390+0000","last_undegraded":"2024-05-11T12:22:43.648390+0000","last_fullsized":"2024-05-11T12:22:43.648390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:57.197028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844877+0000","last_change":"2024-05-11T12:21:11.528072+0000","last_active":"2024-05-11T12:22:43.844877+0000","last_peered":"2024-05-11T12:22:43.844877+0000","last_clean":"2024-05-11T12:22:43.844877+0000","last_became_active":"2024-05-11T12:21:11.527478+0000","last_became_peered":"2024-05-11T12:21:11.527478+0000","last_unstale":"2024-05-11T12:22:43.844877+0000","last_undegraded":"2024-05-11T12:22:43.844877+0000","last_fullsized":"2024-05-11T12:22:43.844877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:57:29.001602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650344+0000","last_change":"2024-05-11T12:21:11.526862+0000","last_active":"2024-05-11T12:22:43.650344+0000","last_peered":"2024-05-11T12:22:43.650344+0000","last_clean":"2024-05-11T12:22:43.650344+0000","last_became_active":"2024-05-11T12:21:11.526553+0000","last_became_peered":"2024-05-11T12:21:11.526553+0000","last_unstale":"2024-05-11T12:22:43.650344+0000","last_undegraded":"2024-05-11T12:22:43.650344+0000","last_fullsized":"2024-05-11T12:22:43.650344+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:48.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650178+0000","last_change":"2024-05-11T12:21:42.669393+0000","last_active":"2024-05-11T12:22:43.650178+0000","last_peered":"2024-05-11T12:22:43.650178+0000","last_clean":"2024-05-11T12:22:43.650178+0000","last_became_active":"2024-05-11T12:21:42.669000+0000","last_became_peered":"2024-05-11T12:21:42.669000+0000","last_unstale":"2024-05-11T12:22:43.650178+0000","last_undegraded":"2024-05-11T12:22:43.650178+0000","last_fullsized":"2024-05-11T12:22:43.650178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:29.439588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891535+0000","last_change":"2024-05-11T12:21:11.540698+0000","last_active":"2024-05-11T12:22:43.891535+0000","last_peered":"2024-05-11T12:22:43.891535+0000","last_clean":"2024-05-11T12:22:43.891535+0000","last_became_active":"2024-05-11T12:21:11.539899+0000","last_became_peered":"2024-05-11T12:21:11.539899+0000","last_unstale":"2024-05-11T12:22:43.891535+0000","last_undegraded":"2024-05-11T12:22:43.891535+0000","last_fullsized":"2024-05-11T12:22:43.891535+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:48.222664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651942+0000","last_change":"2024-05-11T12:21:11.537055+0000","last_active":"2024-05-11T12:22:43.651942+0000","last_peered":"2024-05-11T12:22:43.651942+0000","last_clean":"2024-05-11T12:22:43.651942+0000","last_became_active":"2024-05-11T12:21:11.536625+0000","last_became_peered":"2024-05-11T12:21:11.536625+0000","last_unstale":"2024-05-11T12:22:43.651942+0000","last_undegraded":"2024-05-11T12:22:43.651942+0000","last_fullsized":"2024-05-11T12:22:43.651942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:33.453467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647256+0000","last_change":"2024-05-11T12:21:11.530254+0000","last_active":"2024-05-11T12:22:43.647256+0000","last_peered":"2024-05-11T12:22:43.647256+0000","last_clean":"2024-05-11T12:22:43.647256+0000","last_became_active":"2024-05-11T12:21:11.529890+0000","last_became_peered":"2024-05-11T12:21:11.529890+0000","last_unstale":"2024-05-11T12:22:43.647256+0000","last_undegraded":"2024-05-11T12:22:43.647256+0000","last_fullsized":"2024-05-11T12:22:43.647256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:36.100931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653657+0000","last_change":"2024-05-11T12:21:11.532893+0000","last_active":"2024-05-11T12:22:43.653657+0000","last_peered":"2024-05-11T12:22:43.653657+0000","last_clean":"2024-05-11T12:22:43.653657+0000","last_became_active":"2024-05-11T12:21:11.532539+0000","last_became_peered":"2024-05-11T12:21:11.532539+0000","last_unstale":"2024-05-11T12:22:43.653657+0000","last_undegraded":"2024-05-11T12:22:43.653657+0000","last_fullsized":"2024-05-11T12:22:43.653657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:14:57.888102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'4","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647499+0000","last_change":"2024-05-11T12:21:08.510909+0000","last_active":"2024-05-11T12:22:43.647499+0000","last_peered":"2024-05-11T12:22:43.647499+0000","last_clean":"2024-05-11T12:22:43.647499+0000","last_became_active":"2024-05-11T12:21:08.509986+0000","last_became_peered":"2024-05-11T12:21:08.509986+0000","last_unstale":"2024-05-11T12:22:43.647499+0000","last_undegraded":"2024-05-11T12:22:43.647499+0000","last_fullsized":"2024-05-11T12:22:43.647499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:03:36.430087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790218+0000","last_change":"2024-05-11T12:21:11.532403+0000","last_active":"2024-05-11T12:22:43.790218+0000","last_peered":"2024-05-11T12:22:43.790218+0000","last_clean":"2024-05-11T12:22:43.790218+0000","last_became_active":"2024-05-11T12:21:11.531907+0000","last_became_peered":"2024-05-11T12:21:11.531907+0000","last_unstale":"2024-05-11T12:22:43.790218+0000","last_undegraded":"2024-05-11T12:22:43.790218+0000","last_fullsized":"2024-05-11T12:22:43.790218+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:53:27.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790424+0000","last_change":"2024-05-11T12:21:08.531664+0000","last_active":"2024-05-11T12:22:43.790424+0000","last_peered":"2024-05-11T12:22:43.790424+0000","last_clean":"2024-05-11T12:22:43.790424+0000","last_became_active":"2024-05-11T12:21:08.531319+0000","last_became_peered":"2024-05-11T12:21:08.531319+0000","last_unstale":"2024-05-11T12:22:43.790424+0000","last_undegraded":"2024-05-11T12:22:43.790424+0000","last_fullsized":"2024-05-11T12:22:43.790424+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:16:08.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650677+0000","last_change":"2024-05-11T12:21:11.532132+0000","last_active":"2024-05-11T12:22:43.650677+0000","last_peered":"2024-05-11T12:22:43.650677+0000","last_clean":"2024-05-11T12:22:43.650677+0000","last_became_active":"2024-05-11T12:21:11.529580+0000","last_became_peered":"2024-05-11T12:21:11.529580+0000","last_unstale":"2024-05-11T12:22:43.650677+0000","last_undegraded":"2024-05-11T12:22:43.650677+0000","last_fullsized":"2024-05-11T12:22:43.650677+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:29.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094338+0000","last_change":"2024-05-11T12:21:42.674898+0000","last_active":"2024-05-11T12:22:44.094338+0000","last_peered":"2024-05-11T12:22:44.094338+0000","last_clean":"2024-05-11T12:22:44.094338+0000","last_became_active":"2024-05-11T12:21:42.669853+0000","last_became_peered":"2024-05-11T12:21:42.669853+0000","last_unstale":"2024-05-11T12:22:44.094338+0000","last_undegraded":"2024-05-11T12:22:44.094338+0000","last_fullsized":"2024-05-11T12:22:44.094338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:36.040786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891618+0000","last_change":"2024-05-11T12:21:11.534706+0000","last_active":"2024-05-11T12:22:43.891618+0000","last_peered":"2024-05-11T12:22:43.891618+0000","last_clean":"2024-05-11T12:22:43.891618+0000","last_became_active":"2024-05-11T12:21:11.534301+0000","last_became_peered":"2024-05-11T12:21:11.534301+0000","last_unstale":"2024-05-11T12:22:43.891618+0000","last_undegraded":"2024-05-11T12:22:43.891618+0000","last_fullsized":"2024-05-11T12:22:43.891618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:24.277988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093993+0000","last_change":"2024-05-11T12:21:08.513794+0000","last_active":"2024-05-11T12:22:44.093993+0000","last_peered":"2024-05-11T12:22:44.093993+0000","last_clean":"2024-05-11T12:22:44.093993+0000","last_became_active":"2024-05-11T12:21:08.513383+0000","last_became_peered":"2024-05-11T12:21:08.513383+0000","last_unstale":"2024-05-11T12:22:44.093993+0000","last_undegraded":"2024-05-11T12:22:44.093993+0000","last_fullsized":"2024-05-11T12:22:44.093993+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:50.558886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751890+0000","last_change":"2024-05-11T12:21:11.537147+0000","last_active":"2024-05-11T12:22:43.751890+0000","last_peered":"2024-05-11T12:22:43.751890+0000","last_clean":"2024-05-11T12:22:43.751890+0000","last_became_active":"2024-05-11T12:21:11.536739+0000","last_became_peered":"2024-05-11T12:21:11.536739+0000","last_unstale":"2024-05-11T12:22:43.751890+0000","last_undegraded":"2024-05-11T12:22:43.751890+0000","last_fullsized":"2024-05-11T12:22:43.751890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:00:45.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751556+0000","last_change":"2024-05-11T12:21:08.513494+0000","last_active":"2024-05-11T12:22:43.751556+0000","last_peered":"2024-05-11T12:22:43.751556+0000","last_clean":"2024-05-11T12:22:43.751556+0000","last_became_active":"2024-05-11T12:21:08.513089+0000","last_became_peered":"2024-05-11T12:21:08.513089+0000","last_unstale":"2024-05-11T12:22:43.751556+0000","last_undegraded":"2024-05-11T12:22:43.751556+0000","last_fullsized":"2024-05-11T12:22:43.751556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:48:28.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093686+0000","last_change":"2024-05-11T12:21:11.527827+0000","last_active":"2024-05-11T12:22:44.093686+0000","last_peered":"2024-05-11T12:22:44.093686+0000","last_clean":"2024-05-11T12:22:44.093686+0000","last_became_active":"2024-05-11T12:21:11.527397+0000","last_became_peered":"2024-05-11T12:21:11.527397+0000","last_unstale":"2024-05-11T12:22:44.093686+0000","last_undegraded":"2024-05-11T12:22:44.093686+0000","last_fullsized":"2024-05-11T12:22:44.093686+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:18.710855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748713+0000","last_change":"2024-05-11T12:21:08.513519+0000","last_active":"2024-05-11T12:22:43.748713+0000","last_peered":"2024-05-11T12:22:43.748713+0000","last_clean":"2024-05-11T12:22:43.748713+0000","last_became_active":"2024-05-11T12:21:08.512566+0000","last_became_peered":"2024-05-11T12:21:08.512566+0000","last_unstale":"2024-05-11T12:22:43.748713+0000","last_undegraded":"2024-05-11T12:22:43.748713+0000","last_fullsized":"2024-05-11T12:22:43.748713+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:56.104043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751737+0000","last_change":"2024-05-11T12:21:11.535364+0000","last_active":"2024-05-11T12:22:43.751737+0000","last_peered":"2024-05-11T12:22:43.751737+0000","last_clean":"2024-05-11T12:22:43.751737+0000","last_became_active":"2024-05-11T12:21:11.534988+0000","last_became_peered":"2024-05-11T12:21:11.534988+0000","last_unstale":"2024-05-11T12:22:43.751737+0000","last_undegraded":"2024-05-11T12:22:43.751737+0000","last_fullsized":"2024-05-11T12:22:43.751737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:25:20.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844702+0000","last_change":"2024-05-11T12:21:08.509913+0000","last_active":"2024-05-11T12:22:43.844702+0000","last_peered":"2024-05-11T12:22:43.844702+0000","last_clean":"2024-05-11T12:22:43.844702+0000","last_became_active":"2024-05-11T12:21:08.509498+0000","last_became_peered":"2024-05-11T12:21:08.509498+0000","last_unstale":"2024-05-11T12:22:43.844702+0000","last_undegraded":"2024-05-11T12:22:43.844702+0000","last_fullsized":"2024-05-11T12:22:43.844702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:43.282700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.789945+0000","last_change":"2024-05-11T12:21:11.533108+0000","last_active":"2024-05-11T12:22:43.789945+0000","last_peered":"2024-05-11T12:22:43.789945+0000","last_clean":"2024-05-11T12:22:43.789945+0000","last_became_active":"2024-05-11T12:21:11.532779+0000","last_became_peered":"2024-05-11T12:21:11.532779+0000","last_unstale":"2024-05-11T12:22:43.789945+0000","last_undegraded":"2024-05-11T12:22:43.789945+0000","last_fullsized":"2024-05-11T12:22:43.789945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:55.311786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751266+0000","last_change":"2024-05-11T12:21:08.513355+0000","last_active":"2024-05-11T12:22:43.751266+0000","last_peered":"2024-05-11T12:22:43.751266+0000","last_clean":"2024-05-11T12:22:43.751266+0000","last_became_active":"2024-05-11T12:21:08.512964+0000","last_became_peered":"2024-05-11T12:21:08.512964+0000","last_unstale":"2024-05-11T12:22:43.751266+0000","last_undegraded":"2024-05-11T12:22:43.751266+0000","last_fullsized":"2024-05-11T12:22:43.751266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:12.296865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.646908+0000","last_change":"2024-05-11T12:21:11.525958+0000","last_active":"2024-05-11T12:22:43.646908+0000","last_peered":"2024-05-11T12:22:43.646908+0000","last_clean":"2024-05-11T12:22:43.646908+0000","last_became_active":"2024-05-11T12:21:11.525285+0000","last_became_peered":"2024-05-11T12:21:11.525285+0000","last_unstale":"2024-05-11T12:22:43.646908+0000","last_undegraded":"2024-05-11T12:22:43.646908+0000","last_fullsized":"2024-05-11T12:22:43.646908+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:24:47.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.749125+0000","last_change":"2024-05-11T12:21:42.673378+0000","last_active":"2024-05-11T12:22:43.749125+0000","last_peered":"2024-05-11T12:22:43.749125+0000","last_clean":"2024-05-11T12:22:43.749125+0000","last_became_active":"2024-05-11T12:21:42.672870+0000","last_became_peered":"2024-05-11T12:21:42.672870+0000","last_unstale":"2024-05-11T12:22:43.749125+0000","last_undegraded":"2024-05-11T12:22:43.749125+0000","last_fullsized":"2024-05-11T12:22:43.749125+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:55:41.845162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.892114+0000","last_change":"2024-05-11T12:21:11.538946+0000","last_active":"2024-05-11T12:22:43.892114+0000","last_peered":"2024-05-11T12:22:43.892114+0000","last_clean":"2024-05-11T12:22:43.892114+0000","last_became_active":"2024-05-11T12:21:11.535804+0000","last_became_peered":"2024-05-11T12:21:11.535804+0000","last_unstale":"2024-05-11T12:22:43.892114+0000","last_undegraded":"2024-05-11T12:22:43.892114+0000","last_fullsized":"2024-05-11T12:22:43.892114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:37:16.221144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649318+0000","last_change":"2024-05-11T12:21:42.674203+0000","last_active":"2024-05-11T12:22:43.649318+0000","last_peered":"2024-05-11T12:22:43.649318+0000","last_clean":"2024-05-11T12:22:43.649318+0000","last_became_active":"2024-05-11T12:21:42.673478+0000","last_became_peered":"2024-05-11T12:21:42.673478+0000","last_unstale":"2024-05-11T12:22:43.649318+0000","last_undegraded":"2024-05-11T12:22:43.649318+0000","last_fullsized":"2024-05-11T12:22:43.649318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:20.648528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650740+0000","last_change":"2024-05-11T12:21:11.535346+0000","last_active":"2024-05-11T12:22:43.650740+0000","last_peered":"2024-05-11T12:22:43.650740+0000","last_clean":"2024-05-11T12:22:43.650740+0000","last_became_active":"2024-05-11T12:21:11.530262+0000","last_became_peered":"2024-05-11T12:21:11.530262+0000","last_unstale":"2024-05-11T12:22:43.650740+0000","last_undegraded":"2024-05-11T12:22:43.650740+0000","last_fullsized":"2024-05-11T12:22:43.650740+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:13.169797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'3","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.752053+0000","last_change":"2024-05-11T12:21:08.531598+0000","last_active":"2024-05-11T12:22:43.752053+0000","last_peered":"2024-05-11T12:22:43.752053+0000","last_clean":"2024-05-11T12:22:43.752053+0000","last_became_active":"2024-05-11T12:21:08.531158+0000","last_became_peered":"2024-05-11T12:21:08.531158+0000","last_unstale":"2024-05-11T12:22:43.752053+0000","last_undegraded":"2024-05-11T12:22:43.752053+0000","last_fullsized":"2024-05-11T12:22:43.752053+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:43.528151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094901+0000","last_change":"2024-05-11T12:22:44.094901+0000","last_active":"2024-05-11T12:22:44.094901+0000","last_peered":"2024-05-11T12:22:44.094901+0000","last_clean":"2024-05-11T12:22:44.094901+0000","last_became_active":"2024-05-11T12:22:44.094563+0000","last_became_peered":"2024-05-11T12:22:44.094563+0000","last_unstale":"2024-05-11T12:22:44.094901+0000","last_undegraded":"2024-05-11T12:22:44.094901+0000","last_fullsized":"2024-05-11T12:22:44.094901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:42:41.499957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,4],"acting":[9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"81'28","reported_seq":261,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:03.782398+0000","last_change":"2024-05-11T12:21:08.516737+0000","last_active":"2024-05-11T12:23:03.782398+0000","last_peered":"2024-05-11T12:23:03.782398+0000","last_clean":"2024-05-11T12:23:03.782398+0000","last_became_active":"2024-05-11T12:21:08.516316+0000","last_became_peered":"2024-05-11T12:21:08.516316+0000","last_unstale":"2024-05-11T12:23:03.782398+0000","last_undegraded":"2024-05-11T12:23:03.782398+0000","last_fullsized":"2024-05-11T12:23:03.782398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:25.769713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":203,"num_read_kb":161,"num_write":28,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094134+0000","last_change":"2024-05-11T12:21:11.528553+0000","last_active":"2024-05-11T12:22:44.094134+0000","last_peered":"2024-05-11T12:22:44.094134+0000","last_clean":"2024-05-11T12:22:44.094134+0000","last_became_active":"2024-05-11T12:21:11.528175+0000","last_became_peered":"2024-05-11T12:21:11.528175+0000","last_unstale":"2024-05-11T12:22:44.094134+0000","last_undegraded":"2024-05-11T12:22:44.094134+0000","last_fullsized":"2024-05-11T12:22:44.094134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:46:42.689851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'2","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651038+0000","last_change":"2024-05-11T12:21:42.673360+0000","last_active":"2024-05-11T12:22:43.651038+0000","last_peered":"2024-05-11T12:22:43.651038+0000","last_clean":"2024-05-11T12:22:43.651038+0000","last_became_active":"2024-05-11T12:21:42.664842+0000","last_became_peered":"2024-05-11T12:21:42.664842+0000","last_unstale":"2024-05-11T12:22:43.651038+0000","last_undegraded":"2024-05-11T12:22:43.651038+0000","last_fullsized":"2024-05-11T12:22:43.651038+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:40.271658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844736+0000","last_change":"2024-05-11T12:21:11.533157+0000","last_active":"2024-05-11T12:22:43.844736+0000","last_peered":"2024-05-11T12:22:43.844736+0000","last_clean":"2024-05-11T12:22:43.844736+0000","last_became_active":"2024-05-11T12:21:11.532814+0000","last_became_peered":"2024-05-11T12:21:11.532814+0000","last_unstale":"2024-05-11T12:22:43.844736+0000","last_undegraded":"2024-05-11T12:22:43.844736+0000","last_fullsized":"2024-05-11T12:22:43.844736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:30:56.555384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751534+0000","last_change":"2024-05-11T12:21:08.511907+0000","last_active":"2024-05-11T12:22:43.751534+0000","last_peered":"2024-05-11T12:22:43.751534+0000","last_clean":"2024-05-11T12:22:43.751534+0000","last_became_active":"2024-05-11T12:21:08.511296+0000","last_became_peered":"2024-05-11T12:21:08.511296+0000","last_unstale":"2024-05-11T12:22:43.751534+0000","last_undegraded":"2024-05-11T12:22:43.751534+0000","last_fullsized":"2024-05-11T12:22:43.751534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:43:17.374216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649287+0000","last_change":"2024-05-11T12:21:11.537859+0000","last_active":"2024-05-11T12:22:43.649287+0000","last_peered":"2024-05-11T12:22:43.649287+0000","last_clean":"2024-05-11T12:22:43.649287+0000","last_became_active":"2024-05-11T12:21:11.537448+0000","last_became_peered":"2024-05-11T12:21:11.537448+0000","last_unstale":"2024-05-11T12:22:43.649287+0000","last_undegraded":"2024-05-11T12:22:43.649287+0000","last_fullsized":"2024-05-11T12:22:43.649287+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:09:33.866539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647830+0000","last_change":"2024-05-11T12:21:08.510228+0000","last_active":"2024-05-11T12:22:43.647830+0000","last_peered":"2024-05-11T12:22:43.647830+0000","last_clean":"2024-05-11T12:22:43.647830+0000","last_became_active":"2024-05-11T12:21:08.509671+0000","last_became_peered":"2024-05-11T12:21:08.509671+0000","last_unstale":"2024-05-11T12:22:43.647830+0000","last_undegraded":"2024-05-11T12:22:43.647830+0000","last_fullsized":"2024-05-11T12:22:43.647830+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:06.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649052+0000","last_change":"2024-05-11T12:21:11.530935+0000","last_active":"2024-05-11T12:22:43.649052+0000","last_peered":"2024-05-11T12:22:43.649052+0000","last_clean":"2024-05-11T12:22:43.649052+0000","last_became_active":"2024-05-11T12:21:11.530579+0000","last_became_peered":"2024-05-11T12:21:11.530579+0000","last_unstale":"2024-05-11T12:22:43.649052+0000","last_undegraded":"2024-05-11T12:22:43.649052+0000","last_fullsized":"2024-05-11T12:22:43.649052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:38.296021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651027+0000","last_change":"2024-05-11T12:21:42.681114+0000","last_active":"2024-05-11T12:22:43.651027+0000","last_peered":"2024-05-11T12:22:43.651027+0000","last_clean":"2024-05-11T12:22:43.651027+0000","last_became_active":"2024-05-11T12:21:42.666878+0000","last_became_peered":"2024-05-11T12:21:42.666878+0000","last_unstale":"2024-05-11T12:22:43.651027+0000","last_undegraded":"2024-05-11T12:22:43.651027+0000","last_fullsized":"2024-05-11T12:22:43.651027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:45.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.893287+0000","last_change":"2024-05-11T12:22:43.892989+0000","last_active":"2024-05-11T12:22:43.893287+0000","last_peered":"2024-05-11T12:22:43.893287+0000","last_clean":"2024-05-11T12:22:43.893287+0000","last_became_active":"2024-05-11T12:22:43.892599+0000","last_became_peered":"2024-05-11T12:22:43.892599+0000","last_unstale":"2024-05-11T12:22:43.893287+0000","last_undegraded":"2024-05-11T12:22:43.893287+0000","last_fullsized":"2024-05-11T12:22:43.893287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:46.357968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094125+0000","last_change":"2024-05-11T12:21:08.510504+0000","last_active":"2024-05-11T12:22:44.094125+0000","last_peered":"2024-05-11T12:22:44.094125+0000","last_clean":"2024-05-11T12:22:44.094125+0000","last_became_active":"2024-05-11T12:21:08.510074+0000","last_became_peered":"2024-05-11T12:21:08.510074+0000","last_unstale":"2024-05-11T12:22:44.094125+0000","last_undegraded":"2024-05-11T12:22:44.094125+0000","last_fullsized":"2024-05-11T12:22:44.094125+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:31:49.465252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.753140+0000","last_change":"2024-05-11T12:22:43.753140+0000","last_active":"2024-05-11T12:22:43.753140+0000","last_peered":"2024-05-11T12:22:43.753140+0000","last_clean":"2024-05-11T12:22:43.753140+0000","last_became_active":"2024-05-11T12:22:43.752693+0000","last_became_peered":"2024-05-11T12:22:43.752693+0000","last_unstale":"2024-05-11T12:22:43.753140+0000","last_undegraded":"2024-05-11T12:22:43.753140+0000","last_fullsized":"2024-05-11T12:22:43.753140+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:13.152017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647457+0000","last_change":"2024-05-11T12:21:08.531926+0000","last_active":"2024-05-11T12:22:43.647457+0000","last_peered":"2024-05-11T12:22:43.647457+0000","last_clean":"2024-05-11T12:22:43.647457+0000","last_became_active":"2024-05-11T12:21:08.531548+0000","last_became_peered":"2024-05-11T12:21:08.531548+0000","last_unstale":"2024-05-11T12:22:43.647457+0000","last_undegraded":"2024-05-11T12:22:43.647457+0000","last_fullsized":"2024-05-11T12:22:43.647457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:25:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"79'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748981+0000","last_change":"2024-05-11T12:21:11.528595+0000","last_active":"2024-05-11T12:22:43.748981+0000","last_peered":"2024-05-11T12:22:43.748981+0000","last_clean":"2024-05-11T12:22:43.748981+0000","last_became_active":"2024-05-11T12:21:11.527782+0000","last_became_peered":"2024-05-11T12:21:11.527782+0000","last_unstale":"2024-05-11T12:22:43.748981+0000","last_undegraded":"2024-05-11T12:22:43.748981+0000","last_fullsized":"2024-05-11T12:22:43.748981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:31:00.337497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649582+0000","last_change":"2024-05-11T12:21:08.508093+0000","last_active":"2024-05-11T12:22:43.649582+0000","last_peered":"2024-05-11T12:22:43.649582+0000","last_clean":"2024-05-11T12:22:43.649582+0000","last_became_active":"2024-05-11T12:21:08.507710+0000","last_became_peered":"2024-05-11T12:21:08.507710+0000","last_unstale":"2024-05-11T12:22:43.649582+0000","last_undegraded":"2024-05-11T12:22:43.649582+0000","last_fullsized":"2024-05-11T12:22:43.649582+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:24.273932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748841+0000","last_change":"2024-05-11T12:21:11.526026+0000","last_active":"2024-05-11T12:22:43.748841+0000","last_peered":"2024-05-11T12:22:43.748841+0000","last_clean":"2024-05-11T12:22:43.748841+0000","last_became_active":"2024-05-11T12:21:11.525628+0000","last_became_peered":"2024-05-11T12:21:11.525628+0000","last_unstale":"2024-05-11T12:22:43.748841+0000","last_undegraded":"2024-05-11T12:22:43.748841+0000","last_fullsized":"2024-05-11T12:22:43.748841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:12.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751702+0000","last_change":"2024-05-11T12:21:11.535808+0000","last_active":"2024-05-11T12:22:43.751702+0000","last_peered":"2024-05-11T12:22:43.751702+0000","last_clean":"2024-05-11T12:22:43.751702+0000","last_became_active":"2024-05-11T12:21:11.535204+0000","last_became_peered":"2024-05-11T12:21:11.535204+0000","last_unstale":"2024-05-11T12:22:43.751702+0000","last_undegraded":"2024-05-11T12:22:43.751702+0000","last_fullsized":"2024-05-11T12:22:43.751702+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:36:01.208543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844596+0000","last_change":"2024-05-11T12:21:08.531838+0000","last_active":"2024-05-11T12:22:43.844596+0000","last_peered":"2024-05-11T12:22:43.844596+0000","last_clean":"2024-05-11T12:22:43.844596+0000","last_became_active":"2024-05-11T12:21:08.531491+0000","last_became_peered":"2024-05-11T12:21:08.531491+0000","last_unstale":"2024-05-11T12:22:43.844596+0000","last_undegraded":"2024-05-11T12:22:43.844596+0000","last_fullsized":"2024-05-11T12:22:43.844596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:33.928985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790145+0000","last_change":"2024-05-11T12:21:11.538563+0000","last_active":"2024-05-11T12:22:43.790145+0000","last_peered":"2024-05-11T12:22:43.790145+0000","last_clean":"2024-05-11T12:22:43.790145+0000","last_became_active":"2024-05-11T12:21:11.538200+0000","last_became_peered":"2024-05-11T12:21:11.538200+0000","last_unstale":"2024-05-11T12:22:43.790145+0000","last_undegraded":"2024-05-11T12:22:43.790145+0000","last_fullsized":"2024-05-11T12:22:43.790145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:18.879528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648338+0000","last_change":"2024-05-11T12:21:08.509972+0000","last_active":"2024-05-11T12:22:43.648338+0000","last_peered":"2024-05-11T12:22:43.648338+0000","last_clean":"2024-05-11T12:22:43.648338+0000","last_became_active":"2024-05-11T12:21:08.509531+0000","last_became_peered":"2024-05-11T12:21:08.509531+0000","last_unstale":"2024-05-11T12:22:43.648338+0000","last_undegraded":"2024-05-11T12:22:43.648338+0000","last_fullsized":"2024-05-11T12:22:43.648338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:47:13.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649321+0000","last_change":"2024-05-11T12:19:42.546860+0000","last_active":"2024-05-11T12:22:43.649321+0000","last_peered":"2024-05-11T12:22:43.649321+0000","last_clean":"2024-05-11T12:22:43.649321+0000","last_became_active":"2024-05-11T12:19:42.541449+0000","last_became_peered":"2024-05-11T12:19:42.541449+0000","last_unstale":"2024-05-11T12:22:43.649321+0000","last_undegraded":"2024-05-11T12:22:43.649321+0000","last_fullsized":"2024-05-11T12:22:43.649321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_clean_scrub_stamp":"2024-05-11T12:16:12.381187+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:45.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844576+0000","last_change":"2024-05-11T12:21:11.536770+0000","last_active":"2024-05-11T12:22:43.844576+0000","last_peered":"2024-05-11T12:22:43.844576+0000","last_clean":"2024-05-11T12:22:43.844576+0000","last_became_active":"2024-05-11T12:21:11.536421+0000","last_became_peered":"2024-05-11T12:21:11.536421+0000","last_unstale":"2024-05-11T12:22:43.844576+0000","last_undegraded":"2024-05-11T12:22:43.844576+0000","last_fullsized":"2024-05-11T12:22:43.844576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:42.217398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649287+0000","last_change":"2024-05-11T12:21:42.675283+0000","last_active":"2024-05-11T12:22:43.649287+0000","last_peered":"2024-05-11T12:22:43.649287+0000","last_clean":"2024-05-11T12:22:43.649287+0000","last_became_active":"2024-05-11T12:21:42.674775+0000","last_became_peered":"2024-05-11T12:21:42.674775+0000","last_unstale":"2024-05-11T12:22:43.649287+0000","last_undegraded":"2024-05-11T12:22:43.649287+0000","last_fullsized":"2024-05-11T12:22:43.649287+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:13:30.143988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094167+0000","last_change":"2024-05-11T12:21:08.533462+0000","last_active":"2024-05-11T12:22:44.094167+0000","last_peered":"2024-05-11T12:22:44.094167+0000","last_clean":"2024-05-11T12:22:44.094167+0000","last_became_active":"2024-05-11T12:21:08.533081+0000","last_became_peered":"2024-05-11T12:21:08.533081+0000","last_unstale":"2024-05-11T12:22:44.094167+0000","last_undegraded":"2024-05-11T12:22:44.094167+0000","last_fullsized":"2024-05-11T12:22:44.094167+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:01:48.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844875+0000","last_change":"2024-05-11T12:21:11.539484+0000","last_active":"2024-05-11T12:22:43.844875+0000","last_peered":"2024-05-11T12:22:43.844875+0000","last_clean":"2024-05-11T12:22:43.844875+0000","last_became_active":"2024-05-11T12:21:11.536810+0000","last_became_peered":"2024-05-11T12:21:11.536810+0000","last_unstale":"2024-05-11T12:22:43.844875+0000","last_undegraded":"2024-05-11T12:22:43.844875+0000","last_fullsized":"2024-05-11T12:22:43.844875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:47:41.461766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'10","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647372+0000","last_change":"2024-05-11T12:21:08.510519+0000","last_active":"2024-05-11T12:22:43.647372+0000","last_peered":"2024-05-11T12:22:43.647372+0000","last_clean":"2024-05-11T12:22:43.647372+0000","last_became_active":"2024-05-11T12:21:08.510180+0000","last_became_peered":"2024-05-11T12:21:08.510180+0000","last_unstale":"2024-05-11T12:22:43.647372+0000","last_undegraded":"2024-05-11T12:22:43.647372+0000","last_fullsized":"2024-05-11T12:22:43.647372+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:24.537043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790387+0000","last_change":"2024-05-11T12:21:11.533726+0000","last_active":"2024-05-11T12:22:43.790387+0000","last_peered":"2024-05-11T12:22:43.790387+0000","last_clean":"2024-05-11T12:22:43.790387+0000","last_became_active":"2024-05-11T12:21:11.533393+0000","last_became_peered":"2024-05-11T12:21:11.533393+0000","last_unstale":"2024-05-11T12:22:43.790387+0000","last_undegraded":"2024-05-11T12:22:43.790387+0000","last_fullsized":"2024-05-11T12:22:43.790387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:03.071666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'2","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648325+0000","last_change":"2024-05-11T12:21:09.036622+0000","last_active":"2024-05-11T12:22:43.648325+0000","last_peered":"2024-05-11T12:22:43.648325+0000","last_clean":"2024-05-11T12:22:43.648325+0000","last_became_active":"2024-05-11T12:21:09.036209+0000","last_became_peered":"2024-05-11T12:21:09.036209+0000","last_unstale":"2024-05-11T12:22:43.648325+0000","last_undegraded":"2024-05-11T12:22:43.648325+0000","last_fullsized":"2024-05-11T12:22:43.648325+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:29:58.947588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.846111+0000","last_change":"2024-05-11T12:22:43.846111+0000","last_active":"2024-05-11T12:22:43.846111+0000","last_peered":"2024-05-11T12:22:43.846111+0000","last_clean":"2024-05-11T12:22:43.846111+0000","last_became_active":"2024-05-11T12:22:43.845712+0000","last_became_peered":"2024-05-11T12:22:43.845712+0000","last_unstale":"2024-05-11T12:22:43.846111+0000","last_undegraded":"2024-05-11T12:22:43.846111+0000","last_fullsized":"2024-05-11T12:22:43.846111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:19:47.516917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891550+0000","last_change":"2024-05-11T12:21:08.533435+0000","last_active":"2024-05-11T12:22:43.891550+0000","last_peered":"2024-05-11T12:22:43.891550+0000","last_clean":"2024-05-11T12:22:43.891550+0000","last_became_active":"2024-05-11T12:21:08.533100+0000","last_became_peered":"2024-05-11T12:21:08.533100+0000","last_unstale":"2024-05-11T12:22:43.891550+0000","last_undegraded":"2024-05-11T12:22:43.891550+0000","last_fullsized":"2024-05-11T12:22:43.891550+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:17.619015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093615+0000","last_change":"2024-05-11T12:21:11.526140+0000","last_active":"2024-05-11T12:22:44.093615+0000","last_peered":"2024-05-11T12:22:44.093615+0000","last_clean":"2024-05-11T12:22:44.093615+0000","last_became_active":"2024-05-11T12:21:11.525810+0000","last_became_peered":"2024-05-11T12:21:11.525810+0000","last_unstale":"2024-05-11T12:22:44.093615+0000","last_undegraded":"2024-05-11T12:22:44.093615+0000","last_fullsized":"2024-05-11T12:22:44.093615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:14:04.160379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790069+0000","last_change":"2024-05-11T12:21:08.508031+0000","last_active":"2024-05-11T12:22:43.790069+0000","last_peered":"2024-05-11T12:22:43.790069+0000","last_clean":"2024-05-11T12:22:43.790069+0000","last_became_active":"2024-05-11T12:21:08.507703+0000","last_became_peered":"2024-05-11T12:21:08.507703+0000","last_unstale":"2024-05-11T12:22:43.790069+0000","last_undegraded":"2024-05-11T12:22:43.790069+0000","last_fullsized":"2024-05-11T12:22:43.790069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:05.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651158+0000","last_change":"2024-05-11T12:21:11.528293+0000","last_active":"2024-05-11T12:22:43.651158+0000","last_peered":"2024-05-11T12:22:43.651158+0000","last_clean":"2024-05-11T12:22:43.651158+0000","last_became_active":"2024-05-11T12:21:11.527862+0000","last_became_peered":"2024-05-11T12:21:11.527862+0000","last_unstale":"2024-05-11T12:22:43.651158+0000","last_undegraded":"2024-05-11T12:22:43.651158+0000","last_fullsized":"2024-05-11T12:22:43.651158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:07.335829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653576+0000","last_change":"2024-05-11T12:21:08.510138+0000","last_active":"2024-05-11T12:22:43.653576+0000","last_peered":"2024-05-11T12:22:43.653576+0000","last_clean":"2024-05-11T12:22:43.653576+0000","last_became_active":"2024-05-11T12:21:08.509566+0000","last_became_peered":"2024-05-11T12:21:08.509566+0000","last_unstale":"2024-05-11T12:22:43.653576+0000","last_undegraded":"2024-05-11T12:22:43.653576+0000","last_fullsized":"2024-05-11T12:22:43.653576+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:24:58.174305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.750630+0000","last_change":"2024-05-11T12:22:43.750230+0000","last_active":"2024-05-11T12:22:43.750630+0000","last_peered":"2024-05-11T12:22:43.750630+0000","last_clean":"2024-05-11T12:22:43.750630+0000","last_became_active":"2024-05-11T12:22:43.749836+0000","last_became_peered":"2024-05-11T12:22:43.749836+0000","last_unstale":"2024-05-11T12:22:43.750630+0000","last_undegraded":"2024-05-11T12:22:43.750630+0000","last_fullsized":"2024-05-11T12:22:43.750630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:36.498862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093884+0000","last_change":"2024-05-11T12:21:42.683706+0000","last_active":"2024-05-11T12:22:44.093884+0000","last_peered":"2024-05-11T12:22:44.093884+0000","last_clean":"2024-05-11T12:22:44.093884+0000","last_became_active":"2024-05-11T12:21:42.675791+0000","last_became_peered":"2024-05-11T12:21:42.675791+0000","last_unstale":"2024-05-11T12:22:44.093884+0000","last_undegraded":"2024-05-11T12:22:44.093884+0000","last_fullsized":"2024-05-11T12:22:44.093884+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:13:26.737525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4],"acting":[8,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649661+0000","last_change":"2024-05-11T12:21:11.544153+0000","last_active":"2024-05-11T12:22:43.649661+0000","last_peered":"2024-05-11T12:22:43.649661+0000","last_clean":"2024-05-11T12:22:43.649661+0000","last_became_active":"2024-05-11T12:21:11.543772+0000","last_became_peered":"2024-05-11T12:21:11.543772+0000","last_unstale":"2024-05-11T12:22:43.649661+0000","last_undegraded":"2024-05-11T12:22:43.649661+0000","last_fullsized":"2024-05-11T12:22:43.649661+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:54:30.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653816+0000","last_change":"2024-05-11T12:21:09.036409+0000","last_active":"2024-05-11T12:22:43.653816+0000","last_peered":"2024-05-11T12:22:43.653816+0000","last_clean":"2024-05-11T12:22:43.653816+0000","last_became_active":"2024-05-11T12:21:09.036009+0000","last_became_peered":"2024-05-11T12:21:09.036009+0000","last_unstale":"2024-05-11T12:22:43.653816+0000","last_undegraded":"2024-05-11T12:22:43.653816+0000","last_fullsized":"2024-05-11T12:22:43.653816+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:55:28.736718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649738+0000","last_change":"2024-05-11T12:21:11.542685+0000","last_active":"2024-05-11T12:22:43.649738+0000","last_peered":"2024-05-11T12:22:43.649738+0000","last_clean":"2024-05-11T12:22:43.649738+0000","last_became_active":"2024-05-11T12:21:11.542130+0000","last_became_peered":"2024-05-11T12:21:11.542130+0000","last_unstale":"2024-05-11T12:22:43.649738+0000","last_undegraded":"2024-05-11T12:22:43.649738+0000","last_fullsized":"2024-05-11T12:22:43.649738+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:22.578809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751889+0000","last_change":"2024-05-11T12:21:08.510869+0000","last_active":"2024-05-11T12:22:43.751889+0000","last_peered":"2024-05-11T12:22:43.751889+0000","last_clean":"2024-05-11T12:22:43.751889+0000","last_became_active":"2024-05-11T12:21:08.510510+0000","last_became_peered":"2024-05-11T12:21:08.510510+0000","last_unstale":"2024-05-11T12:22:43.751889+0000","last_undegraded":"2024-05-11T12:22:43.751889+0000","last_fullsized":"2024-05-11T12:22:43.751889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:50.002313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649208+0000","last_change":"2024-05-11T12:21:11.538622+0000","last_active":"2024-05-11T12:22:43.649208+0000","last_peered":"2024-05-11T12:22:43.649208+0000","last_clean":"2024-05-11T12:22:43.649208+0000","last_became_active":"2024-05-11T12:21:11.538229+0000","last_became_peered":"2024-05-11T12:21:11.538229+0000","last_unstale":"2024-05-11T12:22:43.649208+0000","last_undegraded":"2024-05-11T12:22:43.649208+0000","last_fullsized":"2024-05-11T12:22:43.649208+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:01:15.912428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.845041+0000","last_change":"2024-05-11T12:21:42.677421+0000","last_active":"2024-05-11T12:22:43.845041+0000","last_peered":"2024-05-11T12:22:43.845041+0000","last_clean":"2024-05-11T12:22:43.845041+0000","last_became_active":"2024-05-11T12:21:42.672695+0000","last_became_peered":"2024-05-11T12:21:42.672695+0000","last_unstale":"2024-05-11T12:22:43.845041+0000","last_undegraded":"2024-05-11T12:22:43.845041+0000","last_fullsized":"2024-05-11T12:22:43.845041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:30:27.606816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649514+0000","last_change":"2024-05-11T12:21:11.536149+0000","last_active":"2024-05-11T12:22:43.649514+0000","last_peered":"2024-05-11T12:22:43.649514+0000","last_clean":"2024-05-11T12:22:43.649514+0000","last_became_active":"2024-05-11T12:21:11.535763+0000","last_became_peered":"2024-05-11T12:21:11.535763+0000","last_unstale":"2024-05-11T12:22:43.649514+0000","last_undegraded":"2024-05-11T12:22:43.649514+0000","last_fullsized":"2024-05-11T12:22:43.649514+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:46:15.246610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651216+0000","last_change":"2024-05-11T12:21:11.534318+0000","last_active":"2024-05-11T12:22:43.651216+0000","last_peered":"2024-05-11T12:22:43.651216+0000","last_clean":"2024-05-11T12:22:43.651216+0000","last_became_active":"2024-05-11T12:21:11.533949+0000","last_became_peered":"2024-05-11T12:21:11.533949+0000","last_unstale":"2024-05-11T12:22:43.651216+0000","last_undegraded":"2024-05-11T12:22:43.651216+0000","last_fullsized":"2024-05-11T12:22:43.651216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:41:42.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653528+0000","last_change":"2024-05-11T12:21:11.529276+0000","last_active":"2024-05-11T12:22:43.653528+0000","last_peered":"2024-05-11T12:22:43.653528+0000","last_clean":"2024-05-11T12:22:43.653528+0000","last_became_active":"2024-05-11T12:21:11.528857+0000","last_became_peered":"2024-05-11T12:21:11.528857+0000","last_unstale":"2024-05-11T12:22:43.653528+0000","last_undegraded":"2024-05-11T12:22:43.653528+0000","last_fullsized":"2024-05-11T12:22:43.653528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:54.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653362+0000","last_change":"2024-05-11T12:21:11.532342+0000","last_active":"2024-05-11T12:22:43.653362+0000","last_peered":"2024-05-11T12:22:43.653362+0000","last_clean":"2024-05-11T12:22:43.653362+0000","last_became_active":"2024-05-11T12:21:11.531949+0000","last_became_peered":"2024-05-11T12:21:11.531949+0000","last_unstale":"2024-05-11T12:22:43.653362+0000","last_undegraded":"2024-05-11T12:22:43.653362+0000","last_fullsized":"2024-05-11T12:22:43.653362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:23:00.010874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.650629+0000","last_change":"2024-05-11T12:21:11.531547+0000","last_active":"2024-05-11T12:22:43.650629+0000","last_peered":"2024-05-11T12:22:43.650629+0000","last_clean":"2024-05-11T12:22:43.650629+0000","last_became_active":"2024-05-11T12:21:11.529702+0000","last_became_peered":"2024-05-11T12:21:11.529702+0000","last_unstale":"2024-05-11T12:22:43.650629+0000","last_undegraded":"2024-05-11T12:22:43.650629+0000","last_fullsized":"2024-05-11T12:22:43.650629+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:13.857026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844937+0000","last_change":"2024-05-11T12:21:11.531261+0000","last_active":"2024-05-11T12:22:43.844937+0000","last_peered":"2024-05-11T12:22:43.844937+0000","last_clean":"2024-05-11T12:22:43.844937+0000","last_became_active":"2024-05-11T12:21:11.530561+0000","last_became_peered":"2024-05-11T12:21:11.530561+0000","last_unstale":"2024-05-11T12:22:43.844937+0000","last_undegraded":"2024-05-11T12:22:43.844937+0000","last_fullsized":"2024-05-11T12:22:43.844937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:52:07.654417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651599+0000","last_change":"2024-05-11T12:21:11.533255+0000","last_active":"2024-05-11T12:22:43.651599+0000","last_peered":"2024-05-11T12:22:43.651599+0000","last_clean":"2024-05-11T12:22:43.651599+0000","last_became_active":"2024-05-11T12:21:11.532909+0000","last_became_peered":"2024-05-11T12:21:11.532909+0000","last_unstale":"2024-05-11T12:22:43.651599+0000","last_undegraded":"2024-05-11T12:22:43.651599+0000","last_fullsized":"2024-05-11T12:22:43.651599+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:07:39.501482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653835+0000","last_change":"2024-05-11T12:21:11.528856+0000","last_active":"2024-05-11T12:22:43.653835+0000","last_peered":"2024-05-11T12:22:43.653835+0000","last_clean":"2024-05-11T12:22:43.653835+0000","last_became_active":"2024-05-11T12:21:11.528230+0000","last_became_peered":"2024-05-11T12:21:11.528230+0000","last_unstale":"2024-05-11T12:22:43.653835+0000","last_undegraded":"2024-05-11T12:22:43.653835+0000","last_fullsized":"2024-05-11T12:22:43.653835+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:40.429045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790285+0000","last_change":"2024-05-11T12:21:11.534606+0000","last_active":"2024-05-11T12:22:43.790285+0000","last_peered":"2024-05-11T12:22:43.790285+0000","last_clean":"2024-05-11T12:22:43.790285+0000","last_became_active":"2024-05-11T12:21:11.534193+0000","last_became_peered":"2024-05-11T12:21:11.534193+0000","last_unstale":"2024-05-11T12:22:43.790285+0000","last_undegraded":"2024-05-11T12:22:43.790285+0000","last_fullsized":"2024-05-11T12:22:43.790285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:00:12.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751347+0000","last_change":"2024-05-11T12:21:11.525959+0000","last_active":"2024-05-11T12:22:43.751347+0000","last_peered":"2024-05-11T12:22:43.751347+0000","last_clean":"2024-05-11T12:22:43.751347+0000","last_became_active":"2024-05-11T12:21:11.525129+0000","last_became_peered":"2024-05-11T12:21:11.525129+0000","last_unstale":"2024-05-11T12:22:43.751347+0000","last_undegraded":"2024-05-11T12:22:43.751347+0000","last_fullsized":"2024-05-11T12:22:43.751347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:28.828336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790042+0000","last_change":"2024-05-11T12:21:11.538591+0000","last_active":"2024-05-11T12:22:43.790042+0000","last_peered":"2024-05-11T12:22:43.790042+0000","last_clean":"2024-05-11T12:22:43.790042+0000","last_became_active":"2024-05-11T12:21:11.537855+0000","last_became_peered":"2024-05-11T12:21:11.537855+0000","last_unstale":"2024-05-11T12:22:43.790042+0000","last_undegraded":"2024-05-11T12:22:43.790042+0000","last_fullsized":"2024-05-11T12:22:43.790042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:52.625763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891800+0000","last_change":"2024-05-11T12:21:11.541185+0000","last_active":"2024-05-11T12:22:43.891800+0000","last_peered":"2024-05-11T12:22:43.891800+0000","last_clean":"2024-05-11T12:22:43.891800+0000","last_became_active":"2024-05-11T12:21:11.540799+0000","last_became_peered":"2024-05-11T12:21:11.540799+0000","last_unstale":"2024-05-11T12:22:43.891800+0000","last_undegraded":"2024-05-11T12:22:43.891800+0000","last_fullsized":"2024-05-11T12:22:43.891800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:24.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844818+0000","last_change":"2024-05-11T12:21:11.527811+0000","last_active":"2024-05-11T12:22:43.844818+0000","last_peered":"2024-05-11T12:22:43.844818+0000","last_clean":"2024-05-11T12:22:43.844818+0000","last_became_active":"2024-05-11T12:21:11.526603+0000","last_became_peered":"2024-05-11T12:21:11.526603+0000","last_unstale":"2024-05-11T12:22:43.844818+0000","last_undegraded":"2024-05-11T12:22:43.844818+0000","last_fullsized":"2024-05-11T12:22:43.844818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:43.833506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651229+0000","last_change":"2024-05-11T12:21:11.530824+0000","last_active":"2024-05-11T12:22:43.651229+0000","last_peered":"2024-05-11T12:22:43.651229+0000","last_clean":"2024-05-11T12:22:43.651229+0000","last_became_active":"2024-05-11T12:21:11.530486+0000","last_became_peered":"2024-05-11T12:21:11.530486+0000","last_unstale":"2024-05-11T12:22:43.651229+0000","last_undegraded":"2024-05-11T12:22:43.651229+0000","last_fullsized":"2024-05-11T12:22:43.651229+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:47.054822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247249,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":62548,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":209,"num_read_kb":167,"num_write":85,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":270336,"data_stored":187644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678056,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":445299,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874284,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1028096,"data_stored":1097694,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037810,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":445847,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":901120,"data_stored":445869,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":1036332,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593794,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36220,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708932,"statfs":{"total":95995035648,"available":95957946368,"internally_reserved":0,"allocated":897024,"data_stored":445382,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36284,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708868,"statfs":{"total":95995035648,"available":95957880832,"internally_reserved":0,"allocated":962560,"data_stored":507015,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":445355,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":507015,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":445603,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":897024,"data_stored":445287,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":1035932,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:23:09.146 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-11T12:23:09.146 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-11T12:23:09.146 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-11T12:23:09.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph health --format=json 2024-05-11T12:23:09.400 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:09.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:09 smithi047 ceph-mon[30375]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:23:10.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:09 smithi064 ceph-mon[34537]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:23:10.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:09 smithi050 ceph-mon[34072]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:23:10.866 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:10 smithi047 ceph-mon[30375]: pgmap v357: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 -- 172.21.15.47:0/1239882947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974101930 msgr2=0x7f3974101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 --2- 172.21.15.47:0/1239882947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974101930 0x7f3974101d10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f396400b0d0 tx=0x7f396400b3e0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 -- 172.21.15.47:0/1239882947 shutdown_connections 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 --2- 172.21.15.47:0/1239882947 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3974109a10 0x7f3974110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 --2- 172.21.15.47:0/1239882947 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974101930 0x7f3974101d10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 --2- 172.21.15.47:0/1239882947 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39741022e0 0x7f3974109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 -- 172.21.15.47:0/1239882947 >> 172.21.15.47:0/1239882947 conn(0x7f3974078650 msgr2=0x7f39740ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 -- 172.21.15.47:0/1239882947 shutdown_connections 2024-05-11T12:23:11.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 -- 172.21.15.47:0/1239882947 wait complete. 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.088+0000 7f3979b82700 1 Processor -- start 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3979b82700 1 -- start start 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3979b82700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3974101930 0x7f397419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3979b82700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39741022e0 0x7f397419fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3979b82700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974109a10 0x7f3974199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3979b82700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f39741137a0 con 0x7f3974109a10 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3979b82700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3974113620 con 0x7f39741022e0 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3979b82700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3974113920 con 0x7f3974101930 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974109a10 0x7f3974199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974109a10 0x7f3974199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46432/0 (socket says 172.21.15.47:46432) 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 -- 172.21.15.47:0/206177948 learned_addr learned my addr 172.21.15.47:0/206177948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f39737fe700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3974101930 0x7f397419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:11.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3972ffd700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39741022e0 0x7f397419fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 -- 172.21.15.47:0/206177948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3974101930 msgr2=0x7f397419f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3974101930 0x7f397419f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 -- 172.21.15.47:0/206177948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39741022e0 msgr2=0x7f397419fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39741022e0 0x7f397419fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 -- 172.21.15.47:0/206177948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f397419a220 con 0x7f3974109a10 2024-05-11T12:23:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.089+0000 7f3973fff700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974109a10 0x7f3974199ac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f396c00b550 tx=0x7f396c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:11.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.090+0000 7f3970ff9700 1 -- 172.21.15.47:0/206177948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c015020 con 0x7f3974109a10 2024-05-11T12:23:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.090+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f397419a510 con 0x7f3974109a10 2024-05-11T12:23:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.090+0000 7f3970ff9700 1 -- 172.21.15.47:0/206177948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396c011ce0 con 0x7f3974109a10 2024-05-11T12:23:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.090+0000 7f3970ff9700 1 -- 172.21.15.47:0/206177948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c010d40 con 0x7f3974109a10 2024-05-11T12:23:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.090+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39741a67d0 con 0x7f3974109a10 2024-05-11T12:23:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.091+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f397406a240 con 0x7f3974109a10 2024-05-11T12:23:11.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.092+0000 7f3970ff9700 1 -- 172.21.15.47:0/206177948 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f396c018020 con 0x7f3974109a10 2024-05-11T12:23:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.093+0000 7f3970ff9700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f395c0b2910 0x7f395c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.093+0000 7f39737fe700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f395c0b2910 0x7f395c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.093+0000 7f3970ff9700 1 -- 172.21.15.47:0/206177948 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f396c0e1bd0 con 0x7f3974109a10 2024-05-11T12:23:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.093+0000 7f39737fe700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f395c0b2910 0x7f395c0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f396400d010 tx=0x7f396400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:11.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.098+0000 7f3970ff9700 1 -- 172.21.15.47:0/206177948 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f396c01f020 con 0x7f3974109a10 2024-05-11T12:23:11.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:10 smithi064 ceph-mon[34537]: pgmap v357: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:11.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:10 smithi050 ceph-mon[34072]: pgmap v357: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:11.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.360+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f397419b020 con 0x7f3974109a10 2024-05-11T12:23:11.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.360+0000 7f3970ff9700 1 -- 172.21.15.47:0/206177948 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f397419b020 con 0x7f3974109a10 2024-05-11T12:23:11.360 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:11.360 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-11T12:23:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f395c0b2910 msgr2=0x7f395c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f395c0b2910 0x7f395c0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f396400d010 tx=0x7f396400b040 comp rx=0 tx=0).stop 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974109a10 msgr2=0x7f3974199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974109a10 0x7f3974199ac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f396c00b550 tx=0x7f396c00b910 comp rx=0 tx=0).stop 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 shutdown_connections 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3974109a10 0x7f3974199ac0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3974101930 0x7f397419f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f395c0b2910 0x7f395c0b4dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 --2- 172.21.15.47:0/206177948 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39741022e0 0x7f397419fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.362+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 >> 172.21.15.47:0/206177948 conn(0x7f3974078650 msgr2=0x7f39741076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.363+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 shutdown_connections 2024-05-11T12:23:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:11.363+0000 7f3979b82700 1 -- 172.21.15.47:0/206177948 wait complete. 2024-05-11T12:23:12.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:11 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/206177948' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-11T12:23:12.065 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-11T12:23:12.066 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:23:12.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:11 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/206177948' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-11T12:23:12.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:11 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/206177948' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-11T12:23:12.318 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:13.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:12 smithi047 ceph-mon[30375]: pgmap v358: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:13.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:12 smithi047 ceph-mon[30375]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-11T12:23:13.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:12 smithi064 ceph-mon[34537]: pgmap v358: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:13.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:12 smithi064 ceph-mon[34537]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-11T12:23:13.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:12 smithi050 ceph-mon[34072]: pgmap v358: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:13.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:12 smithi050 ceph-mon[34072]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-11T12:23:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 -- 172.21.15.47:0/1361333300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 msgr2=0x7fd1a81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 --2- 172.21.15.47:0/1361333300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 0x7fd1a81103d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000b0a0 tx=0x7fd1a000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:23:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 -- 172.21.15.47:0/1361333300 shutdown_connections 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 --2- 172.21.15.47:0/1361333300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 0x7fd1a81103d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 --2- 172.21.15.47:0/1361333300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a81095b0 0x7fd1a8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 --2- 172.21.15.47:0/1361333300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806bac0 0x7fd1a806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 -- 172.21.15.47:0/1361333300 >> 172.21.15.47:0/1361333300 conn(0x7fd1a80793c0 msgr2=0x7fd1a80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 -- 172.21.15.47:0/1361333300 shutdown_connections 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.991+0000 7fd1ad928700 1 -- 172.21.15.47:0/1361333300 wait complete. 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 Processor -- start 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 -- start start 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806bac0 0x7fd1a8102510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 0x7fd1a8102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a81095b0 0x7fd1a8102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a81136d0 con 0x7fd1a806c480 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8113550 con 0x7fd1a81095b0 2024-05-11T12:23:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1ad928700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8113850 con 0x7fd1a806bac0 2024-05-11T12:23:13.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1a67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 0x7fd1a8102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:13.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806bac0 0x7fd1a8102510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:13.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1a67fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 0x7fd1a8102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46454/0 (socket says 172.21.15.47:46454) 2024-05-11T12:23:13.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1a67fc700 1 -- 172.21.15.47:0/2159772634 learned_addr learned my addr 172.21.15.47:0/2159772634 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.992+0000 7fd1a77fe700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a81095b0 0x7fd1a8102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1a6ffd700 1 -- 172.21.15.47:0/2159772634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a81095b0 msgr2=0x7fd1a8102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1a6ffd700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a81095b0 0x7fd1a8102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1a6ffd700 1 -- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 msgr2=0x7fd1a8102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1a6ffd700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 0x7fd1a8102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1a6ffd700 1 -- 172.21.15.47:0/2159772634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a81a8050 con 0x7fd1a806bac0 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1a6ffd700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806bac0 0x7fd1a8102510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c970 tx=0x7fd19800cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1ac926700 1 -- 172.21.15.47:0/2159772634 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19800ce60 con 0x7fd1a806bac0 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1ac926700 1 -- 172.21.15.47:0/2159772634 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19800f450 con 0x7fd1a806bac0 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a81a8340 con 0x7fd1a806bac0 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1ac926700 1 -- 172.21.15.47:0/2159772634 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198018660 con 0x7fd1a806bac0 2024-05-11T12:23:13.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.993+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a88d0 con 0x7fd1a806bac0 2024-05-11T12:23:13.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.994+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a806a240 con 0x7fd1a806bac0 2024-05-11T12:23:13.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.996+0000 7fd1ac926700 1 -- 172.21.15.47:0/2159772634 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd19800f8e0 con 0x7fd1a806bac0 2024-05-11T12:23:13.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.997+0000 7fd1ac926700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd1900b2900 0x7fd1900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:13.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.997+0000 7fd1a67fc700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd1900b2900 0x7fd1900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:13.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.997+0000 7fd1ac926700 1 -- 172.21.15.47:0/2159772634 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd1980e2280 con 0x7fd1a806bac0 2024-05-11T12:23:13.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:13.997+0000 7fd1a67fc700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd1900b2900 0x7fd1900b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8079950 tx=0x7fd19c007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:14.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.001+0000 7fd1ac926700 1 -- 172.21.15.47:0/2159772634 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd198054df0 con 0x7fd1a806bac0 2024-05-11T12:23:14.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.262+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1a802e570 con 0x7fd1a806bac0 2024-05-11T12:23:14.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.263+0000 7fd1ac926700 1 -- 172.21.15.47:0/2159772634 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fd198014030 con 0x7fd1a806bac0 2024-05-11T12:23:14.263 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:14.263 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd1900b2900 msgr2=0x7fd1900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd1900b2900 0x7fd1900b4dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8079950 tx=0x7fd19c007480 comp rx=0 tx=0).stop 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806bac0 msgr2=0x7fd1a8102510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806bac0 0x7fd1a8102510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd19800c970 tx=0x7fd19800cc80 comp rx=0 tx=0).stop 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 shutdown_connections 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd1900b2900 0x7fd1900b4dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd1a81095b0 0x7fd1a8102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd1a806bac0 0x7fd1a8102510 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 --2- 172.21.15.47:0/2159772634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd1a806c480 0x7fd1a8102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 >> 172.21.15.47:0/2159772634 conn(0x7fd1a80793c0 msgr2=0x7fd1a8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 shutdown_connections 2024-05-11T12:23:14.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:14.266+0000 7fd1ad928700 1 -- 172.21.15.47:0/2159772634 wait complete. 2024-05-11T12:23:14.267 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:23:14.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:14 smithi047 ceph-mon[30375]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:14.966 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:14 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2159772634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:14.995 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:23:15.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:14 smithi064 ceph-mon[34537]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:15.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:14 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2159772634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:15.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:14 smithi050 ceph-mon[34072]: pgmap v359: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:15.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:14 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2159772634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:15.259 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 -- 172.21.15.47:0/1858602267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8101f10 msgr2=0x7f72b8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 --2- 172.21.15.47:0/1858602267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8101f10 0x7f72b8102390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009990 tx=0x7f72a0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 -- 172.21.15.47:0/1858602267 shutdown_connections 2024-05-11T12:23:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 --2- 172.21.15.47:0/1858602267 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b81028d0 0x7f72b8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 --2- 172.21.15.47:0/1858602267 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b8108630 0x7f72b8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:16.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 --2- 172.21.15.47:0/1858602267 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8101f10 0x7f72b8102390 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:16.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 -- 172.21.15.47:0/1858602267 >> 172.21.15.47:0/1858602267 conn(0x7f72b8077770 msgr2=0x7f72b8077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:16.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 -- 172.21.15.47:0/1858602267 shutdown_connections 2024-05-11T12:23:16.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.950+0000 7f72be982700 1 -- 172.21.15.47:0/1858602267 wait complete. 2024-05-11T12:23:16.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 Processor -- start 2024-05-11T12:23:16.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 -- start start 2024-05-11T12:23:16.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b8101f10 0x7f72b819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:16.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b81028d0 0x7f72b819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:16.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8108630 0x7f72b8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:16.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f72b81137c0 con 0x7f72b8108630 2024-05-11T12:23:16.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f72b8113640 con 0x7f72b8101f10 2024-05-11T12:23:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72be982700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f72b8113940 con 0x7f72b81028d0 2024-05-11T12:23:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72bcf1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8108630 0x7f72b8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72bcf1f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8108630 0x7f72b8199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33350/0 (socket says 172.21.15.47:33350) 2024-05-11T12:23:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72bcf1f700 1 -- 172.21.15.47:0/1024984569 learned_addr learned my addr 172.21.15.47:0/1024984569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72b7fff700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b8101f10 0x7f72b819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72affff700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b81028d0 0x7f72b819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:16.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72bcf1f700 1 -- 172.21.15.47:0/1024984569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b81028d0 msgr2=0x7f72b819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:16.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.951+0000 7f72bcf1f700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b81028d0 0x7f72b819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:16.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72bcf1f700 1 -- 172.21.15.47:0/1024984569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b8101f10 msgr2=0x7f72b819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:16.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72bcf1f700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b8101f10 0x7f72b819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:16.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72bcf1f700 1 -- 172.21.15.47:0/1024984569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b819a1e0 con 0x7f72b8108630 2024-05-11T12:23:16.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72affff700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b81028d0 0x7f72b819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:16.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72bcf1f700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8108630 0x7f72b8199a80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f72b0007d70 tx=0x7f72b000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:16.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72b5ffb700 1 -- 172.21.15.47:0/1024984569 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0016020 con 0x7f72b8108630 2024-05-11T12:23:16.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72b5ffb700 1 -- 172.21.15.47:0/1024984569 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b000ad40 con 0x7f72b8108630 2024-05-11T12:23:16.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b819a4d0 con 0x7f72b8108630 2024-05-11T12:23:16.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72b5ffb700 1 -- 172.21.15.47:0/1024984569 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b00115e0 con 0x7f72b8108630 2024-05-11T12:23:16.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.952+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b81a6740 con 0x7f72b8108630 2024-05-11T12:23:16.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.953+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b806a240 con 0x7f72b8108630 2024-05-11T12:23:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.955+0000 7f72b5ffb700 1 -- 172.21.15.47:0/1024984569 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f72b0019020 con 0x7f72b8108630 2024-05-11T12:23:16.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.956+0000 7f72b5ffb700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f72a40b2880 0x7f72a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.956+0000 7f72b7fff700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f72a40b2880 0x7f72a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.956+0000 7f72b5ffb700 1 -- 172.21.15.47:0/1024984569 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f72b00e2660 con 0x7f72b8108630 2024-05-11T12:23:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.957+0000 7f72b7fff700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f72a40b2880 0x7f72a40b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72b8108e70 tx=0x7f72a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:16.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:16.960+0000 7f72b5ffb700 1 -- 172.21.15.47:0/1024984569 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f72b0020020 con 0x7f72b8108630 2024-05-11T12:23:17.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:16 smithi047 ceph-mon[30375]: pgmap v360: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:17.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:16 smithi064 ceph-mon[34537]: pgmap v360: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:17.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:16 smithi050 ceph-mon[34072]: pgmap v360: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.223+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72b802e570 con 0x7f72b8108630 2024-05-11T12:23:17.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.224+0000 7f72b5ffb700 1 -- 172.21.15.47:0/1024984569 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f72b00aa2b0 con 0x7f72b8108630 2024-05-11T12:23:17.224 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:17.225 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f72a40b2880 msgr2=0x7f72a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f72a40b2880 0x7f72a40b4d40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f72b8108e70 tx=0x7f72a800b410 comp rx=0 tx=0).stop 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8108630 msgr2=0x7f72b8199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8108630 0x7f72b8199a80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f72b0007d70 tx=0x7f72b000a340 comp rx=0 tx=0).stop 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 shutdown_connections 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72b8108630 0x7f72b8199a80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f72a40b2880 0x7f72a40b4d40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f72b8101f10 0x7f72b819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 --2- 172.21.15.47:0/1024984569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f72b81028d0 0x7f72b819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 >> 172.21.15.47:0/1024984569 conn(0x7f72b8077770 msgr2=0x7f72b8100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 shutdown_connections 2024-05-11T12:23:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:17.227+0000 7f72be982700 1 -- 172.21.15.47:0/1024984569 wait complete. 2024-05-11T12:23:17.228 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:23:17.899 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:17 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1024984569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:17.929 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:23:18.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:17 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1024984569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:18.188 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:18.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:17 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1024984569' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:19.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:18 smithi047 ceph-mon[30375]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:19.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:18 smithi064 ceph-mon[34537]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:19.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:18 smithi050 ceph-mon[34072]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:19.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 -- 172.21.15.47:0/2845339952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0102740 msgr2=0x7f23c010ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 --2- 172.21.15.47:0/2845339952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0102740 0x7f23c010ea00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23b800b0d0 tx=0x7f23b800b3e0 comp rx=0 tx=0).stop 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 -- 172.21.15.47:0/2845339952 shutdown_connections 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 --2- 172.21.15.47:0/2845339952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0102740 0x7f23c010ea00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 --2- 172.21.15.47:0/2845339952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23c0107be0 0x7f23c0107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 --2- 172.21.15.47:0/2845339952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c0101d80 0x7f23c0102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 -- 172.21.15.47:0/2845339952 >> 172.21.15.47:0/2845339952 conn(0x7f23c0077700 msgr2=0x7f23c0077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 -- 172.21.15.47:0/2845339952 shutdown_connections 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.825+0000 7f23bf59e700 1 -- 172.21.15.47:0/2845339952 wait complete. 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 Processor -- start 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 -- start start 2024-05-11T12:23:19.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23c0101d80 0x7f23c019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c0102740 0x7f23c019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0107be0 0x7f23c01988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0112630 con 0x7f23c0107be0 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01124b0 con 0x7f23c0101d80 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bf59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f23c01127b0 con 0x7f23c0102740 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0107be0 0x7f23c01988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0107be0 0x7f23c01988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33372/0 (socket says 172.21.15.47:33372) 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 -- 172.21.15.47:0/301759678 learned_addr learned my addr 172.21.15.47:0/301759678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23be59c700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23c0101d80 0x7f23c019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bdd9b700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c0102740 0x7f23c019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 -- 172.21.15.47:0/301759678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c0102740 msgr2=0x7f23c019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c0102740 0x7f23c019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:19.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 -- 172.21.15.47:0/301759678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23c0101d80 msgr2=0x7f23c019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23c0101d80 0x7f23c019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.826+0000 7f23bed9d700 1 -- 172.21.15.47:0/301759678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c0199050 con 0x7f23c0107be0 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.827+0000 7f23bed9d700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0107be0 0x7f23c01988f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23b80095c0 tx=0x7f23b8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.827+0000 7f23b77fe700 1 -- 172.21.15.47:0/301759678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b801f070 con 0x7f23c0107be0 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.827+0000 7f23b77fe700 1 -- 172.21.15.47:0/301759678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23b8007e20 con 0x7f23c0107be0 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.827+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01992e0 con 0x7f23c0107be0 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.827+0000 7f23b77fe700 1 -- 172.21.15.47:0/301759678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b801b4a0 con 0x7f23c0107be0 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.827+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a5600 con 0x7f23c0107be0 2024-05-11T12:23:19.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.828+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c0103270 con 0x7f23c0107be0 2024-05-11T12:23:19.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.829+0000 7f23b77fe700 1 -- 172.21.15.47:0/301759678 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f23b8022070 con 0x7f23c0107be0 2024-05-11T12:23:19.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.830+0000 7f23b77fe700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f23ac0b2880 0x7f23ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:19.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.831+0000 7f23be59c700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f23ac0b2880 0x7f23ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:19.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.831+0000 7f23b77fe700 1 -- 172.21.15.47:0/301759678 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f23b800f020 con 0x7f23c0107be0 2024-05-11T12:23:19.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.831+0000 7f23be59c700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f23ac0b2880 0x7f23ac0b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f23c0199a30 tx=0x7f23b0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:19.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:19.834+0000 7f23b77fe700 1 -- 172.21.15.47:0/301759678 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23b8029020 con 0x7f23c0107be0 2024-05-11T12:23:20.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.054+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23c0199e20 con 0x7f23c0107be0 2024-05-11T12:23:20.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.055+0000 7f23b77fe700 1 -- 172.21.15.47:0/301759678 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22917 (secure 0 0 0) 0x7f23b80ab560 con 0x7f23c0107be0 2024-05-11T12:23:20.056 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:20.056 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:22:43.640272+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f23ac0b2880 msgr2=0x7f23ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f23ac0b2880 0x7f23ac0b4d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f23c0199a30 tx=0x7f23b0009190 comp rx=0 tx=0).stop 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0107be0 msgr2=0x7f23c01988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0107be0 0x7f23c01988f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23b80095c0 tx=0x7f23b8007a20 comp rx=0 tx=0).stop 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 shutdown_connections 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f23c0107be0 0x7f23c01988f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f23ac0b2880 0x7f23ac0b4d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f23c0101d80 0x7f23c019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 --2- 172.21.15.47:0/301759678 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f23c0102740 0x7f23c019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:20.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 >> 172.21.15.47:0/301759678 conn(0x7f23c0077700 msgr2=0x7f23c01009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 shutdown_connections 2024-05-11T12:23:20.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:20.058+0000 7f23bf59e700 1 -- 172.21.15.47:0/301759678 wait complete. 2024-05-11T12:23:20.754 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:23:21.007 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:21.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:20 smithi047 ceph-mon[30375]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:21.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:20 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/301759678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:23:21.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:20 smithi064 ceph-mon[34537]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:21.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:20 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/301759678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:23:21.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:20 smithi050 ceph-mon[34072]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:21.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:20 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/301759678' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:23:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 -- 172.21.15.47:0/2564384408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 msgr2=0x7f0fa8108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:22.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/2564384408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 0x7f0fa8108910 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400b0a0 tx=0x7f0fa400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 -- 172.21.15.47:0/2564384408 shutdown_connections 2024-05-11T12:23:22.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/2564384408 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa8102e10 0x7f0fa810f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/2564384408 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 0x7f0fa8108910 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/2564384408 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 0x7f0fa81028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 -- 172.21.15.47:0/2564384408 >> 172.21.15.47:0/2564384408 conn(0x7f0fa8079400 msgr2=0x7f0fa8079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 -- 172.21.15.47:0/2564384408 shutdown_connections 2024-05-11T12:23:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.668+0000 7f0fb00ef700 1 -- 172.21.15.47:0/2564384408 wait complete. 2024-05-11T12:23:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 Processor -- start 2024-05-11T12:23:22.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 -- start start 2024-05-11T12:23:22.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 0x7f0fa819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:22.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa8102e10 0x7f0fa819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 0x7f0fa81a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa81125e0 con 0x7f0fa8108530 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8112460 con 0x7f0fa8102450 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fb00ef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa8112760 con 0x7f0fa8102e10 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fade8b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 0x7f0fa819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fae68c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 0x7f0fa81a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fade8b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 0x7f0fa819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:55704/0 (socket says 172.21.15.47:55704) 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fade8b700 1 -- 172.21.15.47:0/1083791433 learned_addr learned my addr 172.21.15.47:0/1083791433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.669+0000 7f0fad68a700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa8102e10 0x7f0fa819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fae68c700 1 -- 172.21.15.47:0/1083791433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa8102e10 msgr2=0x7f0fa819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fae68c700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa8102e10 0x7f0fa819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fae68c700 1 -- 172.21.15.47:0/1083791433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 msgr2=0x7f0fa819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fae68c700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 0x7f0fa819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fae68c700 1 -- 172.21.15.47:0/1083791433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa81a5fc0 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fade8b700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 0x7f0fa819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fae68c700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 0x7f0fa81a5a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000d720 tx=0x7f0fa000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0f9effd700 1 -- 172.21.15.47:0/1083791433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0016070 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0f9effd700 1 -- 172.21.15.47:0/1083791433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fa000f690 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa81a6250 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0f9effd700 1 -- 172.21.15.47:0/1083791433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0003a40 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.670+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa81a6760 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.671+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa810a9c0 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.672+0000 7f0f9effd700 1 -- 172.21.15.47:0/1083791433 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0fa0003be0 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.674+0000 7f0f9effd700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f940b2900 0x7f0f940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.674+0000 7f0fade8b700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f940b2900 0x7f0f940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.674+0000 7f0f9effd700 1 -- 172.21.15.47:0/1083791433 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0fa00e1ce0 con 0x7f0fa8108530 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.674+0000 7f0fade8b700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f940b2900 0x7f0f940b4dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009e60 tx=0x7f0fa400b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:22.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.677+0000 7f0f9effd700 1 -- 172.21.15.47:0/1083791433 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0fa001f070 con 0x7f0fa8108530 2024-05-11T12:23:22.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.955+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fa810a3a0 con 0x7f0fa8108530 2024-05-11T12:23:22.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:22 smithi047 ceph-mon[30375]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:22.956 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:22 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:23:22.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.955+0000 7f0f9effd700 1 -- 172.21.15.47:0/1083791433 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f0fa00a9930 con 0x7f0fa8108530 2024-05-11T12:23:22.956 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:22.957 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f940b2900 msgr2=0x7f0f940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f940b2900 0x7f0f940b4dc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009e60 tx=0x7f0fa400b7f0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 msgr2=0x7f0fa81a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 0x7f0fa81a5a80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000d720 tx=0x7f0fa000da30 comp rx=0 tx=0).stop 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 shutdown_connections 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0fa8108530 0x7f0fa81a5a80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f940b2900 0x7f0f940b4dc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0fa8102450 0x7f0fa819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 --2- 172.21.15.47:0/1083791433 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0fa8102e10 0x7f0fa819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 >> 172.21.15.47:0/1083791433 conn(0x7f0fa8079400 msgr2=0x7f0fa81012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 shutdown_connections 2024-05-11T12:23:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:22.958+0000 7f0fb00ef700 1 -- 172.21.15.47:0/1083791433 wait complete. 2024-05-11T12:23:22.960 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:23:22.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:22 smithi050 ceph-mon[34072]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:22.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:22 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:23:23.100 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:22 smithi064 ceph-mon[34537]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:23.100 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:22 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:23:23.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:23 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1083791433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:23.827 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-11T12:21:13.603454+0000', 'modified': '2024-05-11T12:21:30.461773+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6835/1314438872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 1314438872}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 1314438872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.50:6837/3384616382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 3384616382}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 3384616382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24485, 'qdb_cluster': [24485]} 2024-05-11T12:23:23.827 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:23:23.827 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-11T12:23:24.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:23 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1083791433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:24.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:23 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1083791433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:24.280 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:24 smithi047 ceph-mon[30375]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:24 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:24 smithi064 ceph-mon[34537]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:25.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:24 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:24 smithi050 ceph-mon[34072]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:25.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:25.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:24 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:26.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 -- 172.21.15.47:0/4235759148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 msgr2=0x7f2af41103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/4235759148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af41103d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009990 tx=0x7f2aec009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 -- 172.21.15.47:0/4235759148 shutdown_connections 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/4235759148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af41103d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/4235759148 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af41095b0 0x7f2af4109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/4235759148 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2af406bac0 0x7f2af406bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 -- 172.21.15.47:0/4235759148 >> 172.21.15.47:0/4235759148 conn(0x7f2af40793c0 msgr2=0x7f2af40797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 -- 172.21.15.47:0/4235759148 shutdown_connections 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.964+0000 7f2afb6aa700 1 -- 172.21.15.47:0/4235759148 wait complete. 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 Processor -- start 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 -- start start 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2af406bac0 0x7f2af419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af41095b0 0x7f2af4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2af4113740 con 0x7f2af406c480 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41135c0 con 0x7f2af406bac0 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2afb6aa700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2af41138c0 con 0x7f2af41095b0 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2af8c45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2af8c45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33400/0 (socket says 172.21.15.47:33400) 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2af8c45700 1 -- 172.21.15.47:0/1450441122 learned_addr learned my addr 172.21.15.47:0/1450441122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.965+0000 7f2af9446700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2af406bac0 0x7f2af419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af9c47700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af41095b0 0x7f2af4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af8c45700 1 -- 172.21.15.47:0/1450441122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af41095b0 msgr2=0x7f2af4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af8c45700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af41095b0 0x7f2af4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af8c45700 1 -- 172.21.15.47:0/1450441122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2af406bac0 msgr2=0x7f2af419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af8c45700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2af406bac0 0x7f2af419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af8c45700 1 -- 172.21.15.47:0/1450441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af419a1e0 con 0x7f2af406c480 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af9c47700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af41095b0 0x7f2af4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:23:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2af8c45700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af419fdb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400b530 tx=0x7f2ae400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2aea7fc700 1 -- 172.21.15.47:0/1450441122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4015020 con 0x7f2af406c480 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af419a4d0 con 0x7f2af406c480 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2aea7fc700 1 -- 172.21.15.47:0/1450441122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ae4011ce0 con 0x7f2af406c480 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2aea7fc700 1 -- 172.21.15.47:0/1450441122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4010d20 con 0x7f2af406c480 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.966+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af41a6740 con 0x7f2af406c480 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.969+0000 7f2aea7fc700 1 -- 172.21.15.47:0/1450441122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2ae4018020 con 0x7f2af406c480 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.970+0000 7f2aea7fc700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2ae00b2880 0x7f2ae00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.970+0000 7f2af9446700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2ae00b2880 0x7f2ae00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.970+0000 7f2aea7fc700 1 -- 172.21.15.47:0/1450441122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2ae40e1b20 con 0x7f2af406c480 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.970+0000 7f2af9446700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2ae00b2880 0x7f2ae00b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2af41090b0 tx=0x7f2af0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.970+0000 7f2b05f40b80 1 Processor -- start 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.970+0000 7f2b05f40b80 1 -- start start 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b4ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609a8a99810 con 0x5609a8b47a10 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5609a8a38770 con 0x5609a8b47050 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5609a8a957d0 con 0x5609a8b4ca80 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b53840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33402/0 (socket says 172.21.15.47:33402) 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 -- 172.21.15.47:0/563209505 learned_addr learned my addr 172.21.15.47:0/563209505 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9446700 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b4ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af8c45700 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 -- 172.21.15.47:0/563209505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 msgr2=0x5609a8b4ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b4ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 -- 172.21.15.47:0/563209505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 msgr2=0x5609a8b474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 -- 172.21.15.47:0/563209505 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a89e6880 con 0x5609a8b47a10 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2af9c47700 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b53840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005f00 tx=0x7f2aec005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e468d7df2cc93fe3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2adf7fe700 1 -- 172.21.15.47:0/563209505 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec026070 con 0x5609a8b47a10 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2adf7fe700 1 -- 172.21.15.47:0/563209505 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec009590 con 0x5609a8b47a10 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.971+0000 7f2adf7fe700 1 -- 172.21.15.47:0/563209505 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00b9d0 con 0x5609a8b47a10 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 -- 172.21.15.47:0/563209505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 msgr2=0x5609a8b53840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b53840 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005f00 tx=0x7f2aec005c00 comp rx=0 tx=0).stop 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 -- 172.21.15.47:0/563209505 shutdown_connections 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b53840 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b4ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/563209505 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 -- 172.21.15.47:0/563209505 >> 172.21.15.47:0/563209505 conn(0x5609a8a73e10 msgr2=0x5609a8b41940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 -- 172.21.15.47:0/563209505 shutdown_connections 2024-05-11T12:23:26.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.972+0000 7f2b05f40b80 1 -- 172.21.15.47:0/563209505 wait complete. 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 -- start start 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 Processor -- start 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b53420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b507c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b53de0 con 0x5609a8b47a10 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5609a89ffbe0 con 0x5609a8b47050 2024-05-11T12:23:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b53f60 con 0x5609a8b4ca80 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2b05f40b80 10 client.0 _ll_register_callbacks cb 0x5609a8a318d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2af8c45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2af8c45700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33412/0 (socket says 172.21.15.47:33412) 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.973+0000 7f2af8c45700 1 -- 172.21.15.47:0/4198756263 learned_addr learned my addr 172.21.15.47:0/4198756263 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af9c47700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af9446700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b53420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af8c45700 1 -- 172.21.15.47:0/4198756263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 msgr2=0x5609a8b50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af8c45700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b50d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af8c45700 1 -- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 msgr2=0x5609a8b53420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af8c45700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b53420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af8c45700 1 -- 172.21.15.47:0/4198756263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a8a35640 con 0x5609a8b47a10 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af9c47700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:26.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af9446700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b53420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2af8c45700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b507c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40df5a0 tx=0x7f2ae40a8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b1aade19cdb659 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2adcff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2adcff9700 1 -- 172.21.15.47:0/4198756263 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40b1ed0 con 0x5609a8b47a10 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aec009590 con 0x5609a8b47a10 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2adcff9700 1 -- 172.21.15.47:0/4198756263 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2ae40a92d0 con 0x5609a8b47a10 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2adcff9700 1 -- 172.21.15.47:0/4198756263 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2ae40a94d0 con 0x5609a8b47a10 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2b05f40b80 10 client.14961 fetch_fsmap learned FSMap version 20 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5609a8b5e670 con 0x5609a8b47a10 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.974+0000 7f2adcff9700 1 -- 172.21.15.47:0/4198756263 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec009590 con 0x5609a8b47a10 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2adcff9700 1 -- 172.21.15.47:0/4198756263 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2ae40a49e0 con 0x5609a8b47a10 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2b05f40b80 10 client.14961 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2b05f40b80 10 client.14961 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2b05f40b80 20 client.14961 populate_metadata read hostname 'smithi047' 2024-05-11T12:23:26.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8b60c50 0x5609a8b620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2b05f40b80 4 client.14961 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609a8aa71f0 con 0x5609a8b60c50 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2af9446700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8b60c50 0x5609a8b620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2af9446700 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8b60c50 0x5609a8b620c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9f5d81beedfe52 server_cookie=63de975745caf2d5 in_seq=0 out_seq=0 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.975+0000 7f2adcff9700 10 client.14961 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2adcff9700 1 -- 172.21.15.47:0/4198756263 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609a8aa71f0 con 0x5609a8b60c50 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2adcff9700 10 client.14961 handle_command_reply: tid=0 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 client.14961 shutdown 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 msgr2=0x5609a8b507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b507c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40df5a0 tx=0x7f2ae40a8cb0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 shutdown_connections 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8b60c50 0x5609a8b620c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b4ca80 0x5609a8b50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b53420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/4198756263 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b47a10 0x5609a8b507c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2adcff9700 0 client.14961 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.976+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 >> 172.21.15.47:0/4198756263 conn(0x5609a8a73e10 msgr2=0x5609a8b3c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.977+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 shutdown_connections 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.977+0000 7f2b05f40b80 1 -- 172.21.15.47:0/4198756263 wait complete. 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.977+0000 7f2b05f40b80 20 client.14961 trim_cache size 0 max 16384 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 Processor -- start 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 -- start start 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b51390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47a10 0x5609a8b4b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b60400 con 0x5609a8b4ca80 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b60ad0 con 0x5609a8b47050 2024-05-11T12:23:26.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b611a0 con 0x5609a8b47a10 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33428/0 (socket says 172.21.15.47:33428) 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 -- 172.21.15.47:0/1359982968 learned_addr learned my addr 172.21.15.47:0/1359982968 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9446700 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b51390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af8c45700 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47a10 0x5609a8b4b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 -- 172.21.15.47:0/1359982968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47a10 msgr2=0x5609a8b4b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47a10 0x5609a8b4b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 -- 172.21.15.47:0/1359982968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 msgr2=0x5609a8b51390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b51390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.998+0000 7f2af9c47700 1 -- 172.21.15.47:0/1359982968 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609a8b3e540 con 0x5609a8b4ca80 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2af9446700 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b51390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2af9c47700 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4beb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009990 tx=0x7f2aec009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91f6a08965ad1a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2adeffd700 1 -- 172.21.15.47:0/1359982968 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec031070 con 0x5609a8b4ca80 2024-05-11T12:23:26.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2adeffd700 1 -- 172.21.15.47:0/1359982968 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec003d10 con 0x5609a8b4ca80 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2adeffd700 1 -- 172.21.15.47:0/1359982968 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec026040 con 0x5609a8b4ca80 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 -- 172.21.15.47:0/1359982968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 msgr2=0x5609a8b4beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4beb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009990 tx=0x7f2aec009290 comp rx=0 tx=0).stop 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 -- 172.21.15.47:0/1359982968 shutdown_connections 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4beb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47050 0x5609a8b51390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/1359982968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47a10 0x5609a8b4b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 -- 172.21.15.47:0/1359982968 >> 172.21.15.47:0/1359982968 conn(0x5609a8a73e10 msgr2=0x5609a8b417d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:25.999+0000 7f2b05f40b80 1 -- 172.21.15.47:0/1359982968 shutdown_connections 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 -- 172.21.15.47:0/1359982968 wait complete. 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 -- start start 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 Processor -- start 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47050 0x5609a8b4d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47a10 0x5609a8b4dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b61f40 con 0x5609a8b4ca80 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b61870 con 0x5609a8b47a10 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5609a8b62610 con 0x5609a8b47050 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2b05f40b80 10 client.0 _ll_register_callbacks cb 0x5609a8a318d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2af9c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2af9c47700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33432/0 (socket says 172.21.15.47:33432) 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2af9c47700 1 -- 172.21.15.47:0/3014506598 learned_addr learned my addr 172.21.15.47:0/3014506598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2af9446700 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47050 0x5609a8b4d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.000+0000 7f2af8c45700 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47a10 0x5609a8b4dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2af9c47700 1 -- 172.21.15.47:0/3014506598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47050 msgr2=0x5609a8b4d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2af9c47700 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47050 0x5609a8b4d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2af9c47700 1 -- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47a10 msgr2=0x5609a8b4dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2af9c47700 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47a10 0x5609a8b4dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2af9c47700 1 -- 172.21.15.47:0/3014506598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609a8b69af0 con 0x5609a8b4ca80 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2af9c47700 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4e190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005f50 tx=0x7f2aec009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbafd6a33bb1e62 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2adcff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2adcff9700 1 -- 172.21.15.47:0/3014506598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec03c060 con 0x5609a8b4ca80 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aec003d10 con 0x5609a8b4ca80 2024-05-11T12:23:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2adcff9700 1 -- 172.21.15.47:0/3014506598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2aec00b670 con 0x5609a8b4ca80 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2adcff9700 1 -- 172.21.15.47:0/3014506598 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2aec031070 con 0x5609a8b4ca80 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2b05f40b80 10 client.14967 fetch_fsmap learned FSMap version 20 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5609a8bcc540 con 0x5609a8b4ca80 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.001+0000 7f2adcff9700 1 -- 172.21.15.47:0/3014506598 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aec003d10 con 0x5609a8b4ca80 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2adcff9700 1 -- 172.21.15.47:0/3014506598 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2aec029030 con 0x5609a8b4ca80 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2b05f40b80 10 client.14967 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2b05f40b80 10 client.14967 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2b05f40b80 20 client.14967 populate_metadata read hostname 'smithi047' 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8bd0990 0x5609a8bcf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2b05f40b80 4 client.14967 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "status"}) -- 0x5609a8aa1290 con 0x5609a8bd0990 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2af9446700 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8bd0990 0x5609a8bcf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2adcff9700 10 client.14967 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.002+0000 7f2af9446700 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8bd0990 0x5609a8bcf6a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72e990c2bcd20bd server_cookie=81cf2d758a289529 in_seq=0 out_seq=0 2024-05-11T12:23:26.081 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:23:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "9b1a9a48-0f8f-11ef-bc98-c7b262605968", 2024-05-11T12:23:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-05-11T12:23:26.081 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24485, 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 131.412679685, 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 20, 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 83, 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 79, 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 162.536472882 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:23:26.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2adcff9700 1 -- 172.21.15.47:0/3014506598 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5609a8aa1290 con 0x5609a8bd0990 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2adcff9700 10 client.14967 handle_command_reply: tid=0 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 client.14967 shutdown 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 msgr2=0x5609a8b4e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4e190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2aec005f50 tx=0x7f2aec009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 shutdown_connections 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5609a8bd0990 0x5609a8bcf6a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5609a8b4ca80 0x5609a8b4e190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5609a8b47050 0x5609a8b4d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 --2- 172.21.15.47:0/3014506598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5609a8b47a10 0x5609a8b4dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 >> 172.21.15.47:0/3014506598 conn(0x5609a8a73e10 msgr2=0x5609a8b4b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2adcff9700 0 client.14967 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 shutdown_connections 2024-05-11T12:23:26.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 1 -- 172.21.15.47:0/3014506598 wait complete. 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.003+0000 7f2b05f40b80 20 client.14967 trim_cache size 0 max 16384 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2ae00b2880 msgr2=0x7f2ae00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2ae00b2880 0x7f2ae00b4d40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2af41090b0 tx=0x7f2af0009040 comp rx=0 tx=0).stop 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 msgr2=0x7f2af419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af419fdb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400b530 tx=0x7f2ae400b8f0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 shutdown_connections 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2af41095b0 0x7f2af4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2ae00b2880 0x7f2ae00b4d40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2af406bac0 0x7f2af419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 --2- 172.21.15.47:0/1450441122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2af406c480 0x7f2af419fdb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 >> 172.21.15.47:0/1450441122 conn(0x7f2af40793c0 msgr2=0x7f2af41010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 shutdown_connections 2024-05-11T12:23:26.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:26.004+0000 7f2afb6aa700 1 -- 172.21.15.47:0/1450441122 wait complete. 2024-05-11T12:23:26.824 INFO:tasks.cephadm:Setup complete, yielding 2024-05-11T12:23:26.825 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-11T12:23:26.834 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-05-11T12:23:26.835 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch status' 2024-05-11T12:23:27.096 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:27.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:26 smithi064 ceph-mon[34537]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:27.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:26 smithi050 ceph-mon[34072]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:27.219 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:26 smithi047 ceph-mon[30375]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:23:28.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:23:28.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:23:28.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:27 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.256 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.256 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.256 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:23:28.256 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:23:28.257 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:27 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:23:28.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:23:28.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:27 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2219243939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01024b0 msgr2=0x7f26d01028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2219243939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01024b0 0x7f26d01028d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0099c0 tx=0x7f26cc009cd0 comp rx=0 tx=0).stop 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2219243939 shutdown_connections 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2219243939 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26d0104050 0x7f26d010a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2219243939 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01024b0 0x7f26d01028d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2219243939 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d01036b0 0x7f26d0103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2219243939 >> 172.21.15.47:0/2219243939 conn(0x7f26d00fda50 msgr2=0x7f26d00ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2219243939 shutdown_connections 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.799+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2219243939 wait complete. 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 Processor -- start 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 -- start start 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26d01024b0 0x7f26d0197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01036b0 0x7f26d01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d0104050 0x7f26d019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26d010dfa0 con 0x7f26d01036b0 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26d010de20 con 0x7f26d01024b0 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d7c0f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26d010e120 con 0x7f26d0104050 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d51aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01036b0 0x7f26d01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d51aa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01036b0 0x7f26d01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35172/0 (socket says 172.21.15.47:35172) 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.800+0000 7f26d51aa700 1 -- 172.21.15.47:0/2627209669 learned_addr learned my addr 172.21.15.47:0/2627209669 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d59ab700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26d01024b0 0x7f26d0197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d61ac700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d0104050 0x7f26d019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:28.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d51aa700 1 -- 172.21.15.47:0/2627209669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d0104050 msgr2=0x7f26d019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d51aa700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d0104050 0x7f26d019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d51aa700 1 -- 172.21.15.47:0/2627209669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26d01024b0 msgr2=0x7f26d0197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d51aa700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26d01024b0 0x7f26d0197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d51aa700 1 -- 172.21.15.47:0/2627209669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d019f290 con 0x7f26d01036b0 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d61ac700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d0104050 0x7f26d019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d59ab700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26d01024b0 0x7f26d0197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d51aa700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01036b0 0x7f26d01981e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26c000e9c0 tx=0x7f26c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26c6ffd700 1 -- 172.21.15.47:0/2627209669 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c000cbd0 con 0x7f26d01036b0 2024-05-11T12:23:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d019f520 con 0x7f26d01036b0 2024-05-11T12:23:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26c6ffd700 1 -- 172.21.15.47:0/2627209669 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26c000cd70 con 0x7f26d01036b0 2024-05-11T12:23:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26c6ffd700 1 -- 172.21.15.47:0/2627209669 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0005610 con 0x7f26d01036b0 2024-05-11T12:23:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.801+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d019fa60 con 0x7f26d01036b0 2024-05-11T12:23:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.803+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d006a240 con 0x7f26d01036b0 2024-05-11T12:23:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.804+0000 7f26c6ffd700 1 -- 172.21.15.47:0/2627209669 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26c0004750 con 0x7f26d01036b0 2024-05-11T12:23:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.805+0000 7f26c6ffd700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26bc0b2880 0x7f26bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.805+0000 7f26d59ab700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26bc0b2880 0x7f26bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.805+0000 7f26c6ffd700 1 -- 172.21.15.47:0/2627209669 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f26c00e2670 con 0x7f26d01036b0 2024-05-11T12:23:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.805+0000 7f26d59ab700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26bc0b2880 0x7f26bc0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b580 tx=0x7f26cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:28.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:28.809+0000 7f26c6ffd700 1 -- 172.21.15.47:0/2627209669 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26c00aa2c0 con 0x7f26d01036b0 2024-05-11T12:23:29.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.041+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f26d006b940 con 0x7f26bc0b2880 2024-05-11T12:23:29.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.042+0000 7f26c6ffd700 1 -- 172.21.15.47:0/2627209669 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f26d006b940 con 0x7f26bc0b2880 2024-05-11T12:23:29.043 INFO:teuthology.orchestra.run.smithi047.stdout:Backend: cephadm 2024-05-11T12:23:29.043 INFO:teuthology.orchestra.run.smithi047.stdout:Available: Yes 2024-05-11T12:23:29.044 INFO:teuthology.orchestra.run.smithi047.stdout:Paused: No 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26bc0b2880 msgr2=0x7f26bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26bc0b2880 0x7f26bc0b4d40 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b580 tx=0x7f26cc005fb0 comp rx=0 tx=0).stop 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01036b0 msgr2=0x7f26d01981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01036b0 0x7f26d01981e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f26c000e9c0 tx=0x7f26c000ecd0 comp rx=0 tx=0).stop 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 shutdown_connections 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26d0104050 0x7f26d019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26bc0b2880 0x7f26bc0b4d40 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26d01024b0 0x7f26d0197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.044+0000 7f26d7c0f700 1 --2- 172.21.15.47:0/2627209669 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26d01036b0 0x7f26d01981e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.045+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 >> 172.21.15.47:0/2627209669 conn(0x7f26d00fda50 msgr2=0x7f26d006b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.045+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 shutdown_connections 2024-05-11T12:23:29.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:29.045+0000 7f26d7c0f700 1 -- 172.21.15.47:0/2627209669 wait complete. 2024-05-11T12:23:29.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:28 smithi047 ceph-mon[30375]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:29.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:28 smithi050 ceph-mon[34072]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:28 smithi064 ceph-mon[34537]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:29.645 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-11T12:23:29.901 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:30.155 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:29 smithi047 ceph-mon[30375]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:30.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:29 smithi050 ceph-mon[34072]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:29 smithi064 ceph-mon[34537]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:31.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:30 smithi047 ceph-mon[30375]: pgmap v367: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:31.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:30 smithi050 ceph-mon[34072]: pgmap v367: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:31.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:30 smithi064 ceph-mon[34537]: pgmap v367: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:31.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 -- 172.21.15.47:0/2598548350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601047d0 msgr2=0x7fb560104c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:31.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 --2- 172.21.15.47:0/2598548350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601047d0 0x7fb560104c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b0d0 tx=0x7fb54800b3e0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 -- 172.21.15.47:0/2598548350 shutdown_connections 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 --2- 172.21.15.47:0/2598548350 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb560105170 0x7fb56010ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 --2- 172.21.15.47:0/2598548350 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5601035d0 0x7fb5601039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 --2- 172.21.15.47:0/2598548350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601047d0 0x7fb560104c30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 -- 172.21.15.47:0/2598548350 >> 172.21.15.47:0/2598548350 conn(0x7fb5600feb70 msgr2=0x7fb560100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 -- 172.21.15.47:0/2598548350 shutdown_connections 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.518+0000 7fb56703f700 1 -- 172.21.15.47:0/2598548350 wait complete. 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 Processor -- start 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 -- start start 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601035d0 0x7fb5601996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5601047d0 0x7fb560199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:31.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb560105170 0x7fb5601a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb56010f070 con 0x7fb5601035d0 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb56010eef0 con 0x7fb5601047d0 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb56703f700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb56010f1f0 con 0x7fb560105170 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601035d0 0x7fb5601996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601035d0 0x7fb5601996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35186/0 (socket says 172.21.15.47:35186) 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 -- 172.21.15.47:0/1519323293 learned_addr learned my addr 172.21.15.47:0/1519323293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb5655dc700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb560105170 0x7fb5601a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb55ffff700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5601047d0 0x7fb560199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 -- 172.21.15.47:0/1519323293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb560105170 msgr2=0x7fb5601a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:31.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb560105170 0x7fb5601a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 -- 172.21.15.47:0/1519323293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5601047d0 msgr2=0x7fb560199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5601047d0 0x7fb560199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.519+0000 7fb564ddb700 1 -- 172.21.15.47:0/1519323293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5601a1340 con 0x7fb5601035d0 2024-05-11T12:23:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.520+0000 7fb564ddb700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601035d0 0x7fb5601996e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb55000d9b0 tx=0x7fb55000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.520+0000 7fb55dffb700 1 -- 172.21.15.47:0/1519323293 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550017030 con 0x7fb5601035d0 2024-05-11T12:23:31.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.520+0000 7fb55dffb700 1 -- 172.21.15.47:0/1519323293 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5500044d0 con 0x7fb5601035d0 2024-05-11T12:23:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.520+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5601a1630 con 0x7fb5601035d0 2024-05-11T12:23:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.520+0000 7fb55dffb700 1 -- 172.21.15.47:0/1519323293 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5500039e0 con 0x7fb5601035d0 2024-05-11T12:23:31.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.520+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5601a1b70 con 0x7fb5601035d0 2024-05-11T12:23:31.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.521+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb56006a240 con 0x7fb5601035d0 2024-05-11T12:23:31.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.522+0000 7fb55dffb700 1 -- 172.21.15.47:0/1519323293 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb55001a070 con 0x7fb5601035d0 2024-05-11T12:23:31.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.523+0000 7fb55dffb700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb54c0b2880 0x7fb54c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:31.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.523+0000 7fb55ffff700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb54c0b2880 0x7fb54c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:31.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.523+0000 7fb55dffb700 1 -- 172.21.15.47:0/1519323293 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb5500e1cd0 con 0x7fb5601035d0 2024-05-11T12:23:31.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.524+0000 7fb55ffff700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb54c0b2880 0x7fb54c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb54800bf60 tx=0x7fb54800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:31.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.528+0000 7fb55dffb700 1 -- 172.21.15.47:0/1519323293 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb5500a98c0 con 0x7fb5601035d0 2024-05-11T12:23:31.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.751+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb560063990 con 0x7fb54c0b2880 2024-05-11T12:23:31.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.765+0000 7fb55dffb700 1 -- 172.21.15.47:0/1519323293 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fb560063990 con 0x7fb54c0b2880 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.a smithi047 running (2m) 4s ago 2m 14.8M - 19.0.0-3601-g30850680 3f6a7beffd02 1790197da40d 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.b smithi050 running (2m) 7s ago 2m 16.1M - 19.0.0-3601-g30850680 3f6a7beffd02 2ccb6b212837 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.c smithi064 running (2m) 7s ago 2m 16.3M - 19.0.0-3601-g30850680 3f6a7beffd02 ace55ea90732 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.d smithi047 running (3m) 4s ago 2m 15.0M - 19.0.0-3601-g30850680 3f6a7beffd02 060ade8db75e 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.e smithi050 running (2m) 7s ago 2m 21.1M - 19.0.0-3601-g30850680 3f6a7beffd02 230b1ad531e2 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.f smithi064 running (2m) 7s ago 2m 18.4M - 19.0.0-3601-g30850680 3f6a7beffd02 8014ca27afe3 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.g smithi047 running (2m) 4s ago 2m 13.9M - 19.0.0-3601-g30850680 3f6a7beffd02 573e378133e2 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.h smithi050 running (2m) 7s ago 2m 26.4M - 19.0.0-3601-g30850680 3f6a7beffd02 7e9c270d0ce3 2024-05-11T12:23:31.766 INFO:teuthology.orchestra.run.smithi047.stdout:mds.i smithi064 running (2m) 7s ago 2m 17.2M - 19.0.0-3601-g30850680 3f6a7beffd02 0ebc0df130aa 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mds.j smithi047 running (3m) 4s ago 3m 14.8M - 19.0.0-3601-g30850680 3f6a7beffd02 f44a52a5393e 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mds.k smithi050 running (3m) 7s ago 3m 14.2M - 19.0.0-3601-g30850680 3f6a7beffd02 77aa021b21a6 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mds.l smithi064 running (3m) 7s ago 3m 16.5M - 19.0.0-3601-g30850680 3f6a7beffd02 9be27e948c6c 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.x smithi047 *:9283,8765 running (10m) 4s ago 10m 530M - 19.0.0-3601-g30850680 3f6a7beffd02 6c6b8a3e0be6 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.y smithi050 *:8443,8765 running (8m) 7s ago 8m 469M - 19.0.0-3601-g30850680 3f6a7beffd02 70f85aab87bf 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mgr.z smithi064 *:8443,8765 running (8m) 7s ago 8m 467M - 19.0.0-3601-g30850680 3f6a7beffd02 9ac350df9f87 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mon.a smithi047 running (10m) 4s ago 10m 54.5M 2048M 19.0.0-3601-g30850680 3f6a7beffd02 da3e44580bab 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mon.b smithi050 running (8m) 7s ago 8m 41.0M 2048M 19.0.0-3601-g30850680 3f6a7beffd02 cddd0d321158 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:mon.c smithi064 running (9m) 7s ago 9m 42.6M 2048M 19.0.0-3601-g30850680 3f6a7beffd02 61ed5fe3c9ef 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 smithi047 running (8m) 4s ago 8m 45.2M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 358aaa24df83 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 smithi050 running (7m) 7s ago 7m 58.9M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 c1ba849aeec5 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 smithi064 running (7m) 7s ago 7m 50.1M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 f32bc7cbd729 2024-05-11T12:23:31.767 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 smithi047 running (7m) 4s ago 7m 43.5M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 45d120627cc9 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 smithi050 running (6m) 7s ago 6m 56.4M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 e327c0773da9 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 smithi064 running (6m) 7s ago 6m 44.5M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 0c6bcc833bea 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 smithi047 running (5m) 4s ago 5m 53.2M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 709c72be3619 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 smithi050 running (5m) 7s ago 5m 56.4M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 3fefd492c163 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 smithi064 running (4m) 7s ago 4m 57.5M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 c2c6f444a384 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 smithi047 running (4m) 4s ago 4m 53.3M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 0b59aee42bc7 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 smithi050 running (3m) 7s ago 3m 56.1M 4273M 19.0.0-3601-g30850680 3f6a7beffd02 e7f729a2edcf 2024-05-11T12:23:31.768 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 smithi064 running (3m) 7s ago 3m 54.2M 4272M 19.0.0-3601-g30850680 3f6a7beffd02 40a6cd66bf73 2024-05-11T12:23:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb54c0b2880 msgr2=0x7fb54c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb54c0b2880 0x7fb54c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb54800bf60 tx=0x7fb54800b040 comp rx=0 tx=0).stop 2024-05-11T12:23:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601035d0 msgr2=0x7fb5601996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601035d0 0x7fb5601996e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb55000d9b0 tx=0x7fb55000dcc0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 shutdown_connections 2024-05-11T12:23:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb560105170 0x7fb5601a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb5601035d0 0x7fb5601996e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb54c0b2880 0x7fb54c0b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 --2- 172.21.15.47:0/1519323293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb5601047d0 0x7fb560199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 >> 172.21.15.47:0/1519323293 conn(0x7fb5600feb70 msgr2=0x7fb560107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 shutdown_connections 2024-05-11T12:23:31.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:31.767+0000 7fb56703f700 1 -- 172.21.15.47:0/1519323293 wait complete. 2024-05-11T12:23:32.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-11T12:23:32.695 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:33.126 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:32 smithi047 ceph-mon[30375]: pgmap v368: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:33.127 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:32 smithi047 ceph-mon[30375]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:33.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:32 smithi050 ceph-mon[34072]: pgmap v368: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:32 smithi050 ceph-mon[34072]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:32 smithi064 ceph-mon[34537]: pgmap v368: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:32 smithi064 ceph-mon[34537]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:34.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 -- 172.21.15.47:0/4188848746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254104040 msgr2=0x7f2254106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:34.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 --2- 172.21.15.47:0/4188848746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254104040 0x7f2254106430 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f224400b0a0 tx=0x7f224400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 -- 172.21.15.47:0/4188848746 shutdown_connections 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 --2- 172.21.15.47:0/4188848746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22541092a0 0x7f225410b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 --2- 172.21.15.47:0/4188848746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254104040 0x7f2254106430 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 --2- 172.21.15.47:0/4188848746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2254106970 0x7f2254108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 -- 172.21.15.47:0/4188848746 >> 172.21.15.47:0/4188848746 conn(0x7f22540fda50 msgr2=0x7f22540ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 -- 172.21.15.47:0/4188848746 shutdown_connections 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.329+0000 7f225bcde700 1 -- 172.21.15.47:0/4188848746 wait complete. 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 Processor -- start 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 -- start start 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2254104040 0x7f225419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254106970 0x7f225419c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22541092a0 0x7f22541a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f225410f180 con 0x7f2254106970 2024-05-11T12:23:34.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f225410f000 con 0x7f2254104040 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f225bcde700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f225410f300 con 0x7f22541092a0 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f2259279700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254106970 0x7f225419c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f2259279700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254106970 0x7f225419c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35204/0 (socket says 172.21.15.47:35204) 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f2259279700 1 -- 172.21.15.47:0/2272797157 learned_addr learned my addr 172.21.15.47:0/2272797157 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.330+0000 7f2259a7a700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2254104040 0x7f225419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f225a27b700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22541092a0 0x7f22541a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2259279700 1 -- 172.21.15.47:0/2272797157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22541092a0 msgr2=0x7f22541a30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2259279700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22541092a0 0x7f22541a30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2259279700 1 -- 172.21.15.47:0/2272797157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2254104040 msgr2=0x7f225419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2259279700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2254104040 0x7f225419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2259279700 1 -- 172.21.15.47:0/2272797157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22541a3790 con 0x7f2254106970 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2259279700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254106970 0x7f225419c070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f224800b560 tx=0x7f224800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:34.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2252ffd700 1 -- 172.21.15.47:0/2272797157 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248015020 con 0x7f2254106970 2024-05-11T12:23:34.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22541a3a80 con 0x7f2254106970 2024-05-11T12:23:34.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2252ffd700 1 -- 172.21.15.47:0/2272797157 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2248011ce0 con 0x7f2254106970 2024-05-11T12:23:34.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f2252ffd700 1 -- 172.21.15.47:0/2272797157 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248010d20 con 0x7f2254106970 2024-05-11T12:23:34.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.331+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22541a4010 con 0x7f2254106970 2024-05-11T12:23:34.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.332+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225406a240 con 0x7f2254106970 2024-05-11T12:23:34.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.334+0000 7f2252ffd700 1 -- 172.21.15.47:0/2272797157 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2248018020 con 0x7f2254106970 2024-05-11T12:23:34.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.335+0000 7f2252ffd700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f223c0b2880 0x7f223c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:34.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.335+0000 7f2259a7a700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f223c0b2880 0x7f223c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:34.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.335+0000 7f2252ffd700 1 -- 172.21.15.47:0/2272797157 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f22480e1ab0 con 0x7f2254106970 2024-05-11T12:23:34.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.335+0000 7f2259a7a700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f223c0b2880 0x7f223c0b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2244009fd0 tx=0x7f224400b7f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:34.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.339+0000 7f2252ffd700 1 -- 172.21.15.47:0/2272797157 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f224801f020 con 0x7f2254106970 2024-05-11T12:23:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.564+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2254063990 con 0x7f223c0b2880 2024-05-11T12:23:34.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.570+0000 7f2252ffd700 1 -- 172.21.15.47:0/2272797157 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2254063990 con 0x7f223c0b2880 2024-05-11T12:23:34.571 INFO:teuthology.orchestra.run.smithi047.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-11T12:23:34.571 INFO:teuthology.orchestra.run.smithi047.stdout:mds.all 12/12 10s ago 3m smithi047=a;smithi047=d;smithi047=g;smithi047=j;smithi050=b;smithi050=e;smithi050=h;smithi050=k;smithi064=c;smithi064=f;smithi064=i;smithi064=l;count:12 2024-05-11T12:23:34.571 INFO:teuthology.orchestra.run.smithi047.stdout:mgr 3/3 10s ago 8m smithi047=x;smithi050=y;smithi064=z;count:3 2024-05-11T12:23:34.571 INFO:teuthology.orchestra.run.smithi047.stdout:mon 3/3 10s ago 9m smithi047:172.21.15.47=a;smithi050:172.21.15.50=b;smithi064:172.21.15.64=c;count:3 2024-05-11T12:23:34.572 INFO:teuthology.orchestra.run.smithi047.stdout:osd 12 10s ago - 2024-05-11T12:23:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f223c0b2880 msgr2=0x7f223c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f223c0b2880 0x7f223c0b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2244009fd0 tx=0x7f224400b7f0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254106970 msgr2=0x7f225419c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254106970 0x7f225419c070 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f224800b560 tx=0x7f224800b870 comp rx=0 tx=0).stop 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 shutdown_connections 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22541092a0 0x7f22541a30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f223c0b2880 0x7f223c0b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2254104040 0x7f225419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 --2- 172.21.15.47:0/2272797157 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254106970 0x7f225419c070 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 >> 172.21.15.47:0/2272797157 conn(0x7f22540fda50 msgr2=0x7f2254103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 shutdown_connections 2024-05-11T12:23:34.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:34.572+0000 7f225bcde700 1 -- 172.21.15.47:0/2272797157 wait complete. 2024-05-11T12:23:35.191 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:34 smithi047 ceph-mon[30375]: pgmap v369: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:35.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:34 smithi050 ceph-mon[34072]: pgmap v369: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:35.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-11T12:23:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:34 smithi064 ceph-mon[34537]: pgmap v369: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:35.475 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:36.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:35 smithi050 ceph-mon[34072]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:36.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:35 smithi047 ceph-mon[30375]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:35 smithi064 ceph-mon[34537]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 -- 172.21.15.47:0/1227056009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 msgr2=0x7f809410b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:37.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 --2- 172.21.15.47:0/1227056009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f809410b710 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f808c00b0a0 tx=0x7f808c00b3b0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 -- 172.21.15.47:0/1227056009 shutdown_connections 2024-05-11T12:23:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 --2- 172.21.15.47:0/1227056009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f809410b710 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 --2- 172.21.15.47:0/1227056009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8094104040 0x7f8094106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 --2- 172.21.15.47:0/1227056009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094106970 0x7f8094108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 -- 172.21.15.47:0/1227056009 >> 172.21.15.47:0/1227056009 conn(0x7f80940fda70 msgr2=0x7f80940ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 -- 172.21.15.47:0/1227056009 shutdown_connections 2024-05-11T12:23:37.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 -- 172.21.15.47:0/1227056009 wait complete. 2024-05-11T12:23:37.165 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:36 smithi047 ceph-mon[30375]: pgmap v370: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.164+0000 7f809b99a700 1 Processor -- start 2024-05-11T12:23:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f809b99a700 1 -- start start 2024-05-11T12:23:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f809b99a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8094104040 0x7f8094199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f809b99a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094106970 0x7f8094199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:37.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f809b99a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f80941a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f809b99a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f809410f0e0 con 0x7f80941092a0 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f809b99a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f809410ef60 con 0x7f8094106970 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f809b99a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f809410f260 con 0x7f8094104040 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f80941a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f80941a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56552/0 (socket says 172.21.15.47:56552) 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 -- 172.21.15.47:0/593116244 learned_addr learned my addr 172.21.15.47:0/593116244 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099736700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8094104040 0x7f8094199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8098f35700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094106970 0x7f8094199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 -- 172.21.15.47:0/593116244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8094104040 msgr2=0x7f8094199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8094104040 0x7f8094199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 -- 172.21.15.47:0/593116244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094106970 msgr2=0x7f8094199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:37.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094106970 0x7f8094199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.165+0000 7f8099f37700 1 -- 172.21.15.47:0/593116244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80941a13b0 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.166+0000 7f8099f37700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f80941a0d10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f808c00b070 tx=0x7f808c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.166+0000 7f808a7fc700 1 -- 172.21.15.47:0/593116244 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c021070 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.166+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80941a1640 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.166+0000 7f808a7fc700 1 -- 172.21.15.47:0/593116244 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f808c007c00 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.166+0000 7f808a7fc700 1 -- 172.21.15.47:0/593116244 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c004810 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.166+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80941a1c00 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.167+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809406a240 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.168+0000 7f808a7fc700 1 -- 172.21.15.47:0/593116244 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f808c003a00 con 0x7f80941092a0 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.169+0000 7f808a7fc700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80800b2880 0x7f80800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:37.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.169+0000 7f8099736700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80800b2880 0x7f80800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:37.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.169+0000 7f808a7fc700 1 -- 172.21.15.47:0/593116244 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f808c00f020 con 0x7f80941092a0 2024-05-11T12:23:37.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.170+0000 7f8099736700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80800b2880 0x7f80800b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8090005950 tx=0x7f809000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:37.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.174+0000 7f808a7fc700 1 -- 172.21.15.47:0/593116244 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f808c02a020 con 0x7f80941092a0 2024-05-11T12:23:37.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:36 smithi050 ceph-mon[34072]: pgmap v370: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:36 smithi064 ceph-mon[34537]: pgmap v370: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:37.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.394+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8094063990 con 0x7f80800b2880 2024-05-11T12:23:37.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.396+0000 7f808a7fc700 1 -- 172.21.15.47:0/593116244 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f8094063990 con 0x7f80800b2880 2024-05-11T12:23:37.395 INFO:teuthology.orchestra.run.smithi047.stdout:HOST ADDR LABELS STATUS 2024-05-11T12:23:37.396 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 172.21.15.47 2024-05-11T12:23:37.396 INFO:teuthology.orchestra.run.smithi047.stdout:smithi050 172.21.15.50 2024-05-11T12:23:37.396 INFO:teuthology.orchestra.run.smithi047.stdout:smithi064 172.21.15.64 2024-05-11T12:23:37.396 INFO:teuthology.orchestra.run.smithi047.stdout:3 hosts in cluster 2024-05-11T12:23:37.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80800b2880 msgr2=0x7f80800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:37.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80800b2880 0x7f80800b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8090005950 tx=0x7f809000b410 comp rx=0 tx=0).stop 2024-05-11T12:23:37.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 msgr2=0x7f80941a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:37.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f80941a0d10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f808c00b070 tx=0x7f808c00ce50 comp rx=0 tx=0).stop 2024-05-11T12:23:37.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 shutdown_connections 2024-05-11T12:23:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f80941092a0 0x7f80941a0d10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8094104040 0x7f8094199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f80800b2880 0x7f80800b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 --2- 172.21.15.47:0/593116244 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8094106970 0x7f8094199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 >> 172.21.15.47:0/593116244 conn(0x7f80940fda70 msgr2=0x7f8094104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 shutdown_connections 2024-05-11T12:23:37.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:37.398+0000 7f809b99a700 1 -- 172.21.15.47:0/593116244 wait complete. 2024-05-11T12:23:38.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-11T12:23:38.319 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:39.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:38 smithi047 ceph-mon[30375]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:39.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:38 smithi047 ceph-mon[30375]: pgmap v371: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:39.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:38 smithi050 ceph-mon[34072]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:38 smithi050 ceph-mon[34072]: pgmap v371: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:38 smithi064 ceph-mon[34537]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:38 smithi064 ceph-mon[34537]: pgmap v371: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 -- 172.21.15.47:0/845343982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c1047b0 msgr2=0x7fa10c104c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 --2- 172.21.15.47:0/845343982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c1047b0 0x7fa10c104c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0099c0 tx=0x7fa0fc009cd0 comp rx=0 tx=0).stop 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 -- 172.21.15.47:0/845343982 shutdown_connections 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 --2- 172.21.15.47:0/845343982 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa10c105150 0x7fa10c10ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 --2- 172.21.15.47:0/845343982 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10c1035b0 0x7fa10c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 --2- 172.21.15.47:0/845343982 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c1047b0 0x7fa10c104c10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 -- 172.21.15.47:0/845343982 >> 172.21.15.47:0/845343982 conn(0x7fa10c0feb50 msgr2=0x7fa10c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 -- 172.21.15.47:0/845343982 shutdown_connections 2024-05-11T12:23:39.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.991+0000 7fa11375d700 1 -- 172.21.15.47:0/845343982 wait complete. 2024-05-11T12:23:39.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 Processor -- start 2024-05-11T12:23:39.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 -- start start 2024-05-11T12:23:39.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10c1035b0 0x7fa10c0772f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa10c1047b0 0x7fa10c075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c105150 0x7fa10c075e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10f020 con 0x7fa10c105150 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10eea0 con 0x7fa10c1047b0 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa11375d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa10c10f1a0 con 0x7fa10c1035b0 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c105150 0x7fa10c075e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c105150 0x7fa10c075e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56574/0 (socket says 172.21.15.47:56574) 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 -- 172.21.15.47:0/657705223 learned_addr learned my addr 172.21.15.47:0/657705223 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa1114f9700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10c1035b0 0x7fa10c0772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa110cf8700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa10c1047b0 0x7fa10c075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 -- 172.21.15.47:0/657705223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10c1035b0 msgr2=0x7fa10c0772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10c1035b0 0x7fa10c0772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 -- 172.21.15.47:0/657705223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa10c1047b0 msgr2=0x7fa10c075940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:39.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa10c1047b0 0x7fa10c075940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa111cfa700 1 -- 172.21.15.47:0/657705223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10c076730 con 0x7fa10c105150 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.992+0000 7fa1114f9700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10c1035b0 0x7fa10c0772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.993+0000 7fa111cfa700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c105150 0x7fa10c075e80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa104007d70 tx=0x7fa10400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.993+0000 7fa1027fc700 1 -- 172.21.15.47:0/657705223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104016020 con 0x7fa10c105150 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.993+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10c1a3820 con 0x7fa10c105150 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.993+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa10c1a3d30 con 0x7fa10c105150 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.993+0000 7fa1027fc700 1 -- 172.21.15.47:0/657705223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa10400ac90 con 0x7fa10c105150 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.993+0000 7fa1027fc700 1 -- 172.21.15.47:0/657705223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104012e60 con 0x7fa10c105150 2024-05-11T12:23:39.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.994+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f0005360 con 0x7fa10c105150 2024-05-11T12:23:39.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.995+0000 7fa1027fc700 1 -- 172.21.15.47:0/657705223 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa104019020 con 0x7fa10c105150 2024-05-11T12:23:39.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.996+0000 7fa1027fc700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0f80b2880 0x7fa0f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:39.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.996+0000 7fa1114f9700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0f80b2880 0x7fa0f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:39.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.996+0000 7fa1027fc700 1 -- 172.21.15.47:0/657705223 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fa1040e20c0 con 0x7fa10c105150 2024-05-11T12:23:39.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.996+0000 7fa1114f9700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0f80b2880 0x7fa0f80b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa10c104610 tx=0x7fa10800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:39.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:39.999+0000 7fa1027fc700 1 -- 172.21.15.47:0/657705223 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa104020030 con 0x7fa10c105150 2024-05-11T12:23:40.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.224+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa0f0000bf0 con 0x7fa0f80b2880 2024-05-11T12:23:40.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.225+0000 7fa1027fc700 1 -- 172.21.15.47:0/657705223 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa0f0000bf0 con 0x7fa0f80b2880 2024-05-11T12:23:40.225 INFO:teuthology.orchestra.run.smithi047.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-11T12:23:40.225 INFO:teuthology.orchestra.run.smithi047.stdout:smithi047 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130022480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-11T12:23:40.225 INFO:teuthology.orchestra.run.smithi047.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-11T12:23:40.225 INFO:teuthology.orchestra.run.smithi047.stdout:smithi064 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT534400C2400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-05-11T12:23:40.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0f80b2880 msgr2=0x7fa0f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:40.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0f80b2880 0x7fa0f80b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa10c104610 tx=0x7fa10800b410 comp rx=0 tx=0).stop 2024-05-11T12:23:40.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c105150 msgr2=0x7fa10c075e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c105150 0x7fa10c075e80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa104007d70 tx=0x7fa10400a340 comp rx=0 tx=0).stop 2024-05-11T12:23:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 shutdown_connections 2024-05-11T12:23:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa10c105150 0x7fa10c075e80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa10c1035b0 0x7fa10c0772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0f80b2880 0x7fa0f80b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:40.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 --2- 172.21.15.47:0/657705223 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa10c1047b0 0x7fa10c075940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:40.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 >> 172.21.15.47:0/657705223 conn(0x7fa10c0feb50 msgr2=0x7fa10c107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:40.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 shutdown_connections 2024-05-11T12:23:40.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:40.228+0000 7fa11375d700 1 -- 172.21.15.47:0/657705223 wait complete. 2024-05-11T12:23:40.933 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-11T12:23:40.942 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi047.front.sepia.ceph.com 2024-05-11T12:23:40.943 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-11T12:23:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:40 smithi050 ceph-mon[34072]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:41.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:40 smithi047 ceph-mon[30375]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:41.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:40 smithi064 ceph-mon[34537]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:42.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:41 smithi050 ceph-mon[34072]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:42.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:41 smithi047 ceph-mon[30375]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:42.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:41 smithi064 ceph-mon[34537]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-11T12:23:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 -- 172.21.15.47:0/3631695088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60102790 msgr2=0x7f6f60102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 --2- 172.21.15.47:0/3631695088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60102790 0x7f6f60102c10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6f50009990 tx=0x7f6f50009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:42.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 -- 172.21.15.47:0/3631695088 shutdown_connections 2024-05-11T12:23:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 --2- 172.21.15.47:0/3631695088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60103150 0x7f6f601035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 --2- 172.21.15.47:0/3631695088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f60101e30 0x7f6f60102250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 --2- 172.21.15.47:0/3631695088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60102790 0x7f6f60102c10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:42.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 -- 172.21.15.47:0/3631695088 >> 172.21.15.47:0/3631695088 conn(0x7f6f600fda50 msgr2=0x7f6f600ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:42.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 -- 172.21.15.47:0/3631695088 shutdown_connections 2024-05-11T12:23:42.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 -- 172.21.15.47:0/3631695088 wait complete. 2024-05-11T12:23:42.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.839+0000 7f6f676ae700 1 Processor -- start 2024-05-11T12:23:42.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f676ae700 1 -- start start 2024-05-11T12:23:42.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f676ae700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60101e30 0x7f6f6019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f676ae700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f60102790 0x7f6f6019be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f676ae700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60103150 0x7f6f601a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f676ae700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6f60111280 con 0x7f6f60103150 2024-05-11T12:23:42.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f676ae700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6f60111100 con 0x7f6f60101e30 2024-05-11T12:23:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f676ae700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6f60111400 con 0x7f6f60102790 2024-05-11T12:23:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60103150 0x7f6f601a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60103150 0x7f6f601a2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56592/0 (socket says 172.21.15.47:56592) 2024-05-11T12:23:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 -- 172.21.15.47:0/2276200092 learned_addr learned my addr 172.21.15.47:0/2276200092 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:42.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f6544a700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60101e30 0x7f6f6019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:42.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f64c49700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f60102790 0x7f6f6019be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:42.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 -- 172.21.15.47:0/2276200092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f60102790 msgr2=0x7f6f6019be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:42.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f60102790 0x7f6f6019be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:42.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 -- 172.21.15.47:0/2276200092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60101e30 msgr2=0x7f6f6019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:42.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60101e30 0x7f6f6019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:42.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 -- 172.21.15.47:0/2276200092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f601a3530 con 0x7f6f60103150 2024-05-11T12:23:42.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.840+0000 7f6f65c4b700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60103150 0x7f6f601a2e90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007da0 tx=0x7f6f5800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:42.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.841+0000 7f6f567fc700 1 -- 172.21.15.47:0/2276200092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58016020 con 0x7f6f60103150 2024-05-11T12:23:42.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.841+0000 7f6f567fc700 1 -- 172.21.15.47:0/2276200092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f5800ac50 con 0x7f6f60103150 2024-05-11T12:23:42.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.841+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f601a3820 con 0x7f6f60103150 2024-05-11T12:23:42.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.841+0000 7f6f567fc700 1 -- 172.21.15.47:0/2276200092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580115e0 con 0x7f6f60103150 2024-05-11T12:23:42.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.841+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f601a3d90 con 0x7f6f60103150 2024-05-11T12:23:42.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.842+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6006a240 con 0x7f6f60103150 2024-05-11T12:23:42.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.843+0000 7f6f567fc700 1 -- 172.21.15.47:0/2276200092 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6f58019020 con 0x7f6f60103150 2024-05-11T12:23:42.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.844+0000 7f6f567fc700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f4c0b2880 0x7f6f4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:42.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.844+0000 7f6f6544a700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f4c0b2880 0x7f6f4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:42.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.844+0000 7f6f567fc700 1 -- 172.21.15.47:0/2276200092 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f6f580e2730 con 0x7f6f60103150 2024-05-11T12:23:42.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.845+0000 7f6f6544a700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f4c0b2880 0x7f6f4c0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009a10 tx=0x7f6f5c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:42.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:42.848+0000 7f6f567fc700 1 -- 172.21.15.47:0/2276200092 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f58020020 con 0x7f6f60103150 2024-05-11T12:23:43.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.109+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f6f6002e5f0 con 0x7f6f60103150 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.110+0000 7f6f567fc700 1 -- 172.21.15.47:0/2276200092 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+2996 (secure 0 0 0) 0x7f6f580aa380 con 0x7f6f60103150 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:e20 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:legacy client fscid: 1 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:Filesystem 'cephfs' (1) 2024-05-11T12:23:43.110 INFO:teuthology.orchestra.run.smithi047.stdout:fs_name cephfs 2024-05-11T12:23:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 20 2024-05-11T12:23:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-11T12:23:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-05-11T12:21:13.603454+0000 2024-05-11T12:23:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-05-11T12:21:30.461773+0000 2024-05-11T12:23:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:tableserver 0 2024-05-11T12:23:43.111 INFO:teuthology.orchestra.run.smithi047.stdout:root 0 2024-05-11T12:23:43.112 INFO:teuthology.orchestra.run.smithi047.stdout:session_timeout 300 2024-05-11T12:23:43.112 INFO:teuthology.orchestra.run.smithi047.stdout:session_autoclose 300 2024-05-11T12:23:43.112 INFO:teuthology.orchestra.run.smithi047.stdout:max_file_size 1099511627776 2024-05-11T12:23:43.112 INFO:teuthology.orchestra.run.smithi047.stdout:max_xattr_size 65536 2024-05-11T12:23:43.112 INFO:teuthology.orchestra.run.smithi047.stdout:required_client_features {} 2024-05-11T12:23:43.112 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure 0 2024-05-11T12:23:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:last_failure_osd_epoch 0 2024-05-11T12:23:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-11T12:23:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:max_mds 1 2024-05-11T12:23:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:in 0 2024-05-11T12:23:43.113 INFO:teuthology.orchestra.run.smithi047.stdout:up {0=24485} 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:failed 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:damaged 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:stopped 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:data_pools [3] 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:metadata_pool 2 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:inline_data disabled 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:balancer 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:bal_rank_mask -1 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:standby_count_wanted 0 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:qdb_cluster leader: 24485 members: 24485 2024-05-11T12:23:43.114 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.h{0:24485} state up:active seq 9 addr [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.115 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.e{0:24467} state up:standby-replay seq 1 addr [v2:172.21.15.50:6834/1314438872,v1:172.21.15.50:6835/1314438872] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:43.115 INFO:teuthology.orchestra.run.smithi047.stdout:Standby daemons: 2024-05-11T12:23:43.115 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:43.115 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.47:6834/3289515287,v1:172.21.15.47:6835/3289515287] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d{-1:14550} state up:standby seq 1 addr [v2:172.21.15.47:6836/2341314538,v1:172.21.15.47:6837/2341314538] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.g{-1:14604} state up:standby seq 1 addr [v2:172.21.15.47:6838/1816706003,v1:172.21.15.47:6839/1816706003] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a{-1:14610} state up:standby seq 1 addr [v2:172.21.15.47:6840/3463326526,v1:172.21.15.47:6841/3463326526] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.l{-1:24388} state up:standby seq 1 addr [v2:172.21.15.64:6832/395408259,v1:172.21.15.64:6833/395408259] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.f{-1:24424} state up:standby seq 1 addr [v2:172.21.15.64:6834/470230978,v1:172.21.15.64:6835/470230978] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c{-1:24436} state up:standby seq 1 addr [v2:172.21.15.64:6836/2460224166,v1:172.21.15.64:6837/2460224166] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.k{-1:24437} state up:standby seq 1 addr [v2:172.21.15.50:6832/995235547,v1:172.21.15.50:6833/995235547] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.i{-1:24445} state up:standby seq 1 addr [v2:172.21.15.64:6838/4099525243,v1:172.21.15.64:6839/4099525243] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.116 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b{-1:24460} state up:standby seq 1 addr [v2:172.21.15.50:6838/1725807153,v1:172.21.15.50:6839/1725807153] compat {c=[1],r=[1],i=[1fff]}] 2024-05-11T12:23:43.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f4c0b2880 msgr2=0x7f6f4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:43.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f4c0b2880 0x7f6f4c0b4d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009a10 tx=0x7f6f5c009190 comp rx=0 tx=0).stop 2024-05-11T12:23:43.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60103150 msgr2=0x7f6f601a2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:43.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60103150 0x7f6f601a2e90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6f58007da0 tx=0x7f6f5800a340 comp rx=0 tx=0).stop 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 shutdown_connections 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6f60103150 0x7f6f601a2e90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6f4c0b2880 0x7f6f4c0b4d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6f60101e30 0x7f6f6019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 --2- 172.21.15.47:0/2276200092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6f60102790 0x7f6f6019be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 >> 172.21.15.47:0/2276200092 conn(0x7f6f600fda50 msgr2=0x7f6f600ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 shutdown_connections 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:43.112+0000 7f6f676ae700 1 -- 172.21.15.47:0/2276200092 wait complete. 2024-05-11T12:23:43.118 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:23:43.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:42 smithi050 ceph-mon[34072]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:43.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:42 smithi047 ceph-mon[30375]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:43.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:42 smithi064 ceph-mon[34537]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:43.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-11T12:23:44.076 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:44.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:43 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2276200092' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-11T12:23:44.199 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:43 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2276200092' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-11T12:23:44.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:43 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2276200092' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-11T12:23:45.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:44 smithi050 ceph-mon[34072]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:45.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:44 smithi047 ceph-mon[30375]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:44 smithi064 ceph-mon[34537]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2175480291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae0101860 msgr2=0x7f5ae0101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2175480291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae0101860 0x7f5ae0101c80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009990 tx=0x7f5ad0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2175480291 shutdown_connections 2024-05-11T12:23:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2175480291 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae01051e0 0x7f5ae010ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2175480291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae0101860 0x7f5ae0101c80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2175480291 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ae0104840 0x7f5ae0104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:45.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2175480291 >> 172.21.15.47:0/2175480291 conn(0x7f5ae00fd1e0 msgr2=0x7f5ae00ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2175480291 shutdown_connections 2024-05-11T12:23:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.896+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2175480291 wait complete. 2024-05-11T12:23:45.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 Processor -- start 2024-05-11T12:23:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 -- start start 2024-05-11T12:23:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae0101860 0x7f5ae01041a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ae0104840 0x7f5ae01027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae01051e0 0x7f5ae0102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae010f020 con 0x7f5ae01051e0 2024-05-11T12:23:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae010eea0 con 0x7f5ae0101860 2024-05-11T12:23:45.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5ae5b8d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae010f1a0 con 0x7f5ae0104840 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5adffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae01051e0 0x7f5ae0102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5adffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae01051e0 0x7f5ae0102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56612/0 (socket says 172.21.15.47:56612) 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5adffff700 1 -- 172.21.15.47:0/2643852476 learned_addr learned my addr 172.21.15.47:0/2643852476 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5adf7fe700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae0101860 0x7f5ae01041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.897+0000 7f5adeffd700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ae0104840 0x7f5ae01027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adffff700 1 -- 172.21.15.47:0/2643852476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ae0104840 msgr2=0x7f5ae01027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adffff700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ae0104840 0x7f5ae01027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adffff700 1 -- 172.21.15.47:0/2643852476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae0101860 msgr2=0x7f5ae01041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adffff700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae0101860 0x7f5ae01041a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adffff700 1 -- 172.21.15.47:0/2643852476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae01035e0 con 0x7f5ae01051e0 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adffff700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae01051e0 0x7f5ae0102d30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800ada0 tx=0x7f5ad800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adcff9700 1 -- 172.21.15.47:0/2643852476 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8017020 con 0x7f5ae01051e0 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae01a38b0 con 0x7f5ae01051e0 2024-05-11T12:23:45.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adcff9700 1 -- 172.21.15.47:0/2643852476 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad800cd80 con 0x7f5ae01051e0 2024-05-11T12:23:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.898+0000 7f5adcff9700 1 -- 172.21.15.47:0/2643852476 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8005260 con 0x7f5ae01051e0 2024-05-11T12:23:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.899+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae01a3dc0 con 0x7f5ae01051e0 2024-05-11T12:23:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.900+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae006a240 con 0x7f5ae01051e0 2024-05-11T12:23:45.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.900+0000 7f5adcff9700 1 -- 172.21.15.47:0/2643852476 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ad800c8d0 con 0x7f5ae01051e0 2024-05-11T12:23:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.901+0000 7f5adcff9700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ac80b2880 0x7f5ac80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.901+0000 7f5adf7fe700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ac80b2880 0x7f5ac80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.901+0000 7f5adcff9700 1 -- 172.21.15.47:0/2643852476 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f5ad80e1a10 con 0x7f5ae01051e0 2024-05-11T12:23:45.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.902+0000 7f5adf7fe700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ac80b2880 0x7f5ac80b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00038a0 tx=0x7f5ad0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:45.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:45.906+0000 7f5adcff9700 1 -- 172.21.15.47:0/2643852476 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ad8021020 con 0x7f5ae01051e0 2024-05-11T12:23:46.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.124+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f5ae0064160 con 0x7f5ae01051e0 2024-05-11T12:23:46.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.126+0000 7f5adcff9700 1 -- 172.21.15.47:0/2643852476 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5452 (secure 0 0 0) 0x7f5ad80a96f0 con 0x7f5ae01051e0 2024-05-11T12:23:46.125 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 83 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-05-11T12:12:45.657702+0000 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:modified 2024-05-11T12:22:43.640272+0000 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:crush_version 26 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:full_ratio 0.95 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:backfillfull_ratio 0.9 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:nearfull_ratio 0.85 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:require_min_compat_client luminous 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:min_compat_client luminous 2024-05-11T12:23:46.126 INFO:teuthology.orchestra.run.smithi047.stdout:require_osd_release squid 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:stretch_mode_enabled false 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:max_osd 12 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6802/105259815,v1:172.21.15.47:6803/105259815] [v2:172.21.15.47:6804/105259815,v1:172.21.15.47:6805/105259815] exists,up f335b8d2-f243-4dbe-862c-98d13f68228f 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.1 up in weight 1 up_from 13 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6800/4057338653,v1:172.21.15.50:6801/4057338653] [v2:172.21.15.50:6802/4057338653,v1:172.21.15.50:6803/4057338653] exists,up 1bacadec-81bc-4e48-b2f7-67645202bba0 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.2 up in weight 1 up_from 18 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6800/765449938,v1:172.21.15.64:6801/765449938] [v2:172.21.15.64:6802/765449938,v1:172.21.15.64:6803/765449938] exists,up 7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.3 up in weight 1 up_from 26 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6810/425091101,v1:172.21.15.47:6811/425091101] [v2:172.21.15.47:6812/425091101,v1:172.21.15.47:6813/425091101] exists,up da71e86e-e235-47c9-bb5e-ebe25a5f6900 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.4 up in weight 1 up_from 32 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6808/3426956641,v1:172.21.15.50:6809/3426956641] [v2:172.21.15.50:6810/3426956641,v1:172.21.15.50:6811/3426956641] exists,up a94dfdef-2e27-4f70-b821-19770c72d177 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.5 up in weight 1 up_from 37 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6808/1906914370,v1:172.21.15.64:6809/1906914370] [v2:172.21.15.64:6810/1906914370,v1:172.21.15.64:6811/1906914370] exists,up 46f116be-c1a5-4e7b-b43c-3a2e79660066 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.6 up in weight 1 up_from 43 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6818/3223062314,v1:172.21.15.47:6819/3223062314] [v2:172.21.15.47:6820/3223062314,v1:172.21.15.47:6821/3223062314] exists,up 0f2ec271-0e78-490a-86f4-c7058de78f6b 2024-05-11T12:23:46.127 INFO:teuthology.orchestra.run.smithi047.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6816/1770681841,v1:172.21.15.50:6817/1770681841] [v2:172.21.15.50:6818/1770681841,v1:172.21.15.50:6819/1770681841] exists,up b473c499-1ea3-4f0c-b534-07b725db8c14 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:osd.8 up in weight 1 up_from 54 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6816/2722012694,v1:172.21.15.64:6817/2722012694] [v2:172.21.15.64:6818/2722012694,v1:172.21.15.64:6819/2722012694] exists,up e67c89b1-f9de-46be-b19d-c60f72ce48f0 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:osd.9 up in weight 1 up_from 60 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.47:6826/2171109969,v1:172.21.15.47:6827/2171109969] [v2:172.21.15.47:6828/2171109969,v1:172.21.15.47:6829/2171109969] exists,up 000a3e9c-c28a-46e4-b668-b809b80e11cd 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:osd.10 up in weight 1 up_from 65 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.50:6824/1444033681,v1:172.21.15.50:6825/1444033681] [v2:172.21.15.50:6826/1444033681,v1:172.21.15.50:6827/1444033681] exists,up a5dc1c98-29f5-4afb-8c44-7c2085f9c414 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:osd.11 up in weight 1 up_from 71 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.64:6824/368350106,v1:172.21.15.64:6825/368350106] [v2:172.21.15.64:6826/368350106,v1:172.21.15.64:6827/368350106] exists,up 8acbed28-84a5-4506-9bb4-ff7e65e8701b 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.0 [8,4] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.b [7,4] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.e [2,1] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.10 [8,11] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.13 [2,11] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.16 [2,4] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.17 [5,1] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 2.1d [2,6] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.6 [3,6] 2024-05-11T12:23:46.128 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.b [3,9] 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.e [4,7] 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.11 [5,9] 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.14 [1,9] 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.31 [4,8] 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.39 [5,8] 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:pg_upmap_items 3.3a [4,6] 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/3174346748 expires 2024-05-12T12:13:41.429763+0000 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/3174346748 expires 2024-05-12T12:13:41.429763+0000 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/892003243 expires 2024-05-12T12:13:41.429763+0000 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/1613929613 expires 2024-05-12T12:13:41.429763+0000 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/4061249581 expires 2024-05-12T12:13:13.704186+0000 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/4250695572 expires 2024-05-12T12:13:13.704186+0000 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/2978137437 expires 2024-05-12T12:13:13.704186+0000 2024-05-11T12:23:46.129 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6800/2872683180 expires 2024-05-12T12:13:13.704186+0000 2024-05-11T12:23:46.130 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:6801/2872683180 expires 2024-05-12T12:13:13.704186+0000 2024-05-11T12:23:46.130 INFO:teuthology.orchestra.run.smithi047.stdout:blocklist 172.21.15.47:0/684328265 expires 2024-05-12T12:13:41.429763+0000 2024-05-11T12:23:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ac80b2880 msgr2=0x7f5ac80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ac80b2880 0x7f5ac80b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00038a0 tx=0x7f5ad0005fb0 comp rx=0 tx=0).stop 2024-05-11T12:23:46.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae01051e0 msgr2=0x7f5ae0102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae01051e0 0x7f5ae0102d30 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800ada0 tx=0x7f5ad800c330 comp rx=0 tx=0).stop 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 shutdown_connections 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ae01051e0 0x7f5ae0102d30 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5ac80b2880 0x7f5ac80b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ae0101860 0x7f5ae01041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 --2- 172.21.15.47:0/2643852476 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ae0104840 0x7f5ae01027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.128+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 >> 172.21.15.47:0/2643852476 conn(0x7f5ae00fd1e0 msgr2=0x7f5ae0107d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.129+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 shutdown_connections 2024-05-11T12:23:46.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:46.129+0000 7f5ae5b8d700 1 -- 172.21.15.47:0/2643852476 wait complete. 2024-05-11T12:23:47.043 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-11T12:23:47.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:23:47.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:46 smithi050 ceph-mon[34072]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:47.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:46 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2643852476' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-11T12:23:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:46 smithi047 ceph-mon[30375]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:46 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2643852476' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-11T12:23:47.308 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:46 smithi064 ceph-mon[34537]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:46 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2643852476' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 -- 172.21.15.47:0/1750957159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102480 msgr2=0x7f7dc8102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/1750957159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102480 0x7f7dc8102900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7db8009990 tx=0x7f7db8009ca0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 -- 172.21.15.47:0/1750957159 shutdown_connections 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/1750957159 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dc8102e40 0x7f7dc810f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/1750957159 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dc81084c0 0x7f7dc81088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/1750957159 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102480 0x7f7dc8102900 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 -- 172.21.15.47:0/1750957159 >> 172.21.15.47:0/1750957159 conn(0x7f7dc8079400 msgr2=0x7f7dc8079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 -- 172.21.15.47:0/1750957159 shutdown_connections 2024-05-11T12:23:49.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.019+0000 7f7dd02c4700 1 -- 172.21.15.47:0/1750957159 wait complete. 2024-05-11T12:23:49.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 Processor -- start 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 -- start start 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dc8102480 0x7f7dc819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102e40 0x7f7dc819b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dc81084c0 0x7f7dc81a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc8112720 con 0x7f7dc8102e40 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc81125a0 con 0x7f7dc8102480 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dd02c4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc81128a0 con 0x7f7dc81084c0 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102e40 0x7f7dc819b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102e40 0x7f7dc819b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59578/0 (socket says 172.21.15.47:59578) 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 -- 172.21.15.47:0/3741481507 learned_addr learned my addr 172.21.15.47:0/3741481507 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dce861700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dc81084c0 0x7f7dc81a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dce060700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dc8102480 0x7f7dc819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 -- 172.21.15.47:0/3741481507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dc81084c0 msgr2=0x7f7dc81a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dc81084c0 0x7f7dc81a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 -- 172.21.15.47:0/3741481507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dc8102480 msgr2=0x7f7dc819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dc8102480 0x7f7dc819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.020+0000 7f7dcd85f700 1 -- 172.21.15.47:0/3741481507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc81a61a0 con 0x7f7dc8102e40 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.021+0000 7f7dce861700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dc81084c0 0x7f7dc81a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.021+0000 7f7dcd85f700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102e40 0x7f7dc819b120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7db8005910 tx=0x7f7db8003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.021+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/3741481507 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8027070 con 0x7f7dc8102e40 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.021+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/3741481507 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db8004140 con 0x7f7dc8102e40 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.021+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc81989d0 con 0x7f7dc8102e40 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.021+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/3741481507 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db800f460 con 0x7f7dc8102e40 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.021+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc8198f00 con 0x7f7dc8102e40 2024-05-11T12:23:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.022+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc806a240 con 0x7f7dc8102e40 2024-05-11T12:23:49.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.024+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/3741481507 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7db802a070 con 0x7f7dc8102e40 2024-05-11T12:23:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.025+0000 7f7dbf7fe700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7db40b2900 0x7f7db40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.025+0000 7f7dce060700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7db40b2900 0x7f7db40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.025+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/3741481507 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f7db80e3970 con 0x7f7dc8102e40 2024-05-11T12:23:49.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.026+0000 7f7dce060700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7db40b2900 0x7f7db40b4dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4006fd0 tx=0x7f7dc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.029+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/3741481507 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7db8031070 con 0x7f7dc8102e40 2024-05-11T12:23:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:48 smithi050 ceph-mon[34072]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:49.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:48 smithi047 ceph-mon[30375]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.288+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dc806c290 con 0x7f7dc8102e40 2024-05-11T12:23:49.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.289+0000 7f7dbf7fe700 1 -- 172.21.15.47:0/3741481507 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f7db8013070 con 0x7f7dc8102e40 2024-05-11T12:23:49.289 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:49.289 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7db40b2900 msgr2=0x7f7db40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7db40b2900 0x7f7db40b4dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4006fd0 tx=0x7f7dc4008040 comp rx=0 tx=0).stop 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102e40 msgr2=0x7f7dc819b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102e40 0x7f7dc819b120 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7db8005910 tx=0x7f7db8003730 comp rx=0 tx=0).stop 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 shutdown_connections 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7dc81084c0 0x7f7dc81a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7db40b2900 0x7f7db40b4dc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7dc8102480 0x7f7dc819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 --2- 172.21.15.47:0/3741481507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7dc8102e40 0x7f7dc819b120 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 >> 172.21.15.47:0/3741481507 conn(0x7f7dc8079400 msgr2=0x7f7dc8100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 shutdown_connections 2024-05-11T12:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:49.291+0000 7f7dd02c4700 1 -- 172.21.15.47:0/3741481507 wait complete. 2024-05-11T12:23:49.292 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:23:49.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:48 smithi064 ceph-mon[34537]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:49.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:49 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3741481507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:49.959 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:23:50.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:49 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3741481507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:50.214 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:50.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:49 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3741481507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:51.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:50 smithi050 ceph-mon[34072]: pgmap v377: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:51.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:50 smithi047 ceph-mon[30375]: pgmap v377: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:50 smithi064 ceph-mon[34537]: pgmap v377: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:51.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 -- 172.21.15.47:0/2839702097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 msgr2=0x7f29d01081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 --2- 172.21.15.47:0/2839702097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d01081d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29c40099c0 tx=0x7f29c4009cd0 comp rx=0 tx=0).stop 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 -- 172.21.15.47:0/2839702097 shutdown_connections 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 --2- 172.21.15.47:0/2839702097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d0108900 0x7f29d0110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 --2- 172.21.15.47:0/2839702097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d0103800 0x7f29d0103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 --2- 172.21.15.47:0/2839702097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d01081d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 -- 172.21.15.47:0/2839702097 >> 172.21.15.47:0/2839702097 conn(0x7f29d007a6b0 msgr2=0x7f29d007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 -- 172.21.15.47:0/2839702097 shutdown_connections 2024-05-11T12:23:51.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.960+0000 7f29d5225700 1 -- 172.21.15.47:0/2839702097 wait complete. 2024-05-11T12:23:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 Processor -- start 2024-05-11T12:23:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 -- start start 2024-05-11T12:23:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d0103800 0x7f29d019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:51.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d0108900 0x7f29d01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01136c0 con 0x7f29d01041b0 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0113540 con 0x7f29d0108900 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29d5225700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29d0113840 con 0x7f29d0103800 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29ce59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59588/0 (socket says 172.21.15.47:59588) 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29ce59c700 1 -- 172.21.15.47:0/1352781465 learned_addr learned my addr 172.21.15.47:0/1352781465 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.961+0000 7f29ced9d700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d0103800 0x7f29d019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:51.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29cf59e700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d0108900 0x7f29d01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29ce59c700 1 -- 172.21.15.47:0/1352781465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d0103800 msgr2=0x7f29d019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29ce59c700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d0103800 0x7f29d019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29ce59c700 1 -- 172.21.15.47:0/1352781465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d0108900 msgr2=0x7f29d01999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29ce59c700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d0108900 0x7f29d01999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29ce59c700 1 -- 172.21.15.47:0/1352781465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d019a120 con 0x7f29d01041b0 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29ced9d700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d0103800 0x7f29d019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29ce59c700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d019fcf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f29c400b580 tx=0x7f29c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29b7fff700 1 -- 172.21.15.47:0/1352781465 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4027070 con 0x7f29d01041b0 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29b7fff700 1 -- 172.21.15.47:0/1352781465 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29c4004020 con 0x7f29d01041b0 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d019a3b0 con 0x7f29d01041b0 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29b7fff700 1 -- 172.21.15.47:0/1352781465 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c4022e30 con 0x7f29d01041b0 2024-05-11T12:23:51.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.962+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01a6680 con 0x7f29d01041b0 2024-05-11T12:23:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.963+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d0104c90 con 0x7f29d01041b0 2024-05-11T12:23:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.965+0000 7f29b7fff700 1 -- 172.21.15.47:0/1352781465 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f29c4022590 con 0x7f29d01041b0 2024-05-11T12:23:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.966+0000 7f29b7fff700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29b80b2880 0x7f29b80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:51.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.966+0000 7f29ced9d700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29b80b2880 0x7f29b80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.966+0000 7f29b7fff700 1 -- 172.21.15.47:0/1352781465 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f29c40e2f90 con 0x7f29d01041b0 2024-05-11T12:23:51.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.966+0000 7f29ced9d700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29b80b2880 0x7f29b80b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005950 tx=0x7f29c00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:51.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:51.970+0000 7f29b7fff700 1 -- 172.21.15.47:0/1352781465 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29c4013070 con 0x7f29d01041b0 2024-05-11T12:23:52.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.234+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29d002e570 con 0x7f29d01041b0 2024-05-11T12:23:52.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.234+0000 7f29b7fff700 1 -- 172.21.15.47:0/1352781465 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f29c40aac70 con 0x7f29d01041b0 2024-05-11T12:23:52.234 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:52.235 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29b80b2880 msgr2=0x7f29b80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29b80b2880 0x7f29b80b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29c0005950 tx=0x7f29c00058e0 comp rx=0 tx=0).stop 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 msgr2=0x7f29d019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d019fcf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f29c400b580 tx=0x7f29c40047b0 comp rx=0 tx=0).stop 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 shutdown_connections 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29b80b2880 0x7f29b80b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f29d0108900 0x7f29d01999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f29d0103800 0x7f29d019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 --2- 172.21.15.47:0/1352781465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f29d01041b0 0x7f29d019fcf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:52.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 >> 172.21.15.47:0/1352781465 conn(0x7f29d007a6b0 msgr2=0x7f29d0109a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:52.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 shutdown_connections 2024-05-11T12:23:52.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:52.237+0000 7f29d5225700 1 -- 172.21.15.47:0/1352781465 wait complete. 2024-05-11T12:23:52.238 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:23:52.934 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:23:53.194 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:53.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:52 smithi050 ceph-mon[34072]: pgmap v378: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:53.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:52 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1352781465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:53.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:52 smithi047 ceph-mon[30375]: pgmap v378: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:53.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:52 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1352781465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:53.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:52 smithi064 ceph-mon[34537]: pgmap v378: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:53.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:52 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1352781465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:23:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 -- 172.21.15.47:0/3456725742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c108900 msgr2=0x7fc11c110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3456725742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c108900 0x7fc11c110480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc11400b0d0 tx=0x7fc11400b3e0 comp rx=0 tx=0).stop 2024-05-11T12:23:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 -- 172.21.15.47:0/3456725742 shutdown_connections 2024-05-11T12:23:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3456725742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c108900 0x7fc11c110480 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:54.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3456725742 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c103800 0x7fc11c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3456725742 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11c1041b0 0x7fc11c1081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 -- 172.21.15.47:0/3456725742 >> 172.21.15.47:0/3456725742 conn(0x7fc11c07a6b0 msgr2=0x7fc11c07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 -- 172.21.15.47:0/3456725742 shutdown_connections 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.821+0000 7fc122cf3700 1 -- 172.21.15.47:0/3456725742 wait complete. 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 Processor -- start 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 -- start start 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c103800 0x7fc11c075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11c1041b0 0x7fc11c075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c108900 0x7fc11c0762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c1136f0 con 0x7fc11c103800 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c113570 con 0x7fc11c1041b0 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc122cf3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc11c113870 con 0x7fc11c108900 2024-05-11T12:23:54.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc120a8f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c103800 0x7fc11c075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc120a8f700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c103800 0x7fc11c075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59618/0 (socket says 172.21.15.47:59618) 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc120a8f700 1 -- 172.21.15.47:0/3773041442 learned_addr learned my addr 172.21.15.47:0/3773041442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc113fff700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11c1041b0 0x7fc11c075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.822+0000 7fc121290700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c108900 0x7fc11c0762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc120a8f700 1 -- 172.21.15.47:0/3773041442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c108900 msgr2=0x7fc11c0762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc120a8f700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c108900 0x7fc11c0762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc120a8f700 1 -- 172.21.15.47:0/3773041442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11c1041b0 msgr2=0x7fc11c075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc120a8f700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11c1041b0 0x7fc11c075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc120a8f700 1 -- 172.21.15.47:0/3773041442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc11c1a8050 con 0x7fc11c103800 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc113fff700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11c1041b0 0x7fc11c075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc120a8f700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c103800 0x7fc11c075850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc11800d980 tx=0x7fc11800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc111ffb700 1 -- 172.21.15.47:0/3773041442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118017070 con 0x7fc11c103800 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc11c1a8340 con 0x7fc11c103800 2024-05-11T12:23:54.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc111ffb700 1 -- 172.21.15.47:0/3773041442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc118004830 con 0x7fc11c103800 2024-05-11T12:23:54.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc111ffb700 1 -- 172.21.15.47:0/3773041442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc118005440 con 0x7fc11c103800 2024-05-11T12:23:54.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.823+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc11c1a8830 con 0x7fc11c103800 2024-05-11T12:23:54.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.824+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc11c10bad0 con 0x7fc11c103800 2024-05-11T12:23:54.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.825+0000 7fc111ffb700 1 -- 172.21.15.47:0/3773041442 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc1180055e0 con 0x7fc11c103800 2024-05-11T12:23:54.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.826+0000 7fc111ffb700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1040b2880 0x7fc1040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:54.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.827+0000 7fc113fff700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1040b2880 0x7fc1040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:54.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.827+0000 7fc111ffb700 1 -- 172.21.15.47:0/3773041442 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc1180e22f0 con 0x7fc11c103800 2024-05-11T12:23:54.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.827+0000 7fc113fff700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1040b2880 0x7fc1040b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc10c005fd0 tx=0x7fc10c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:54.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:54.831+0000 7fc111ffb700 1 -- 172.21.15.47:0/3773041442 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc118020020 con 0x7fc11c103800 2024-05-11T12:23:55.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.050+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc11c10b4b0 con 0x7fc11c103800 2024-05-11T12:23:55.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.052+0000 7fc111ffb700 1 -- 172.21.15.47:0/3773041442 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22917 (secure 0 0 0) 0x7fc1180a9ee0 con 0x7fc11c103800 2024-05-11T12:23:55.052 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:23:55.052 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:22:43.640272+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1040b2880 msgr2=0x7fc1040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1040b2880 0x7fc1040b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc10c005fd0 tx=0x7fc10c005db0 comp rx=0 tx=0).stop 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c103800 msgr2=0x7fc11c075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c103800 0x7fc11c075850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc11800d980 tx=0x7fc11800dd40 comp rx=0 tx=0).stop 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 shutdown_connections 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc11c108900 0x7fc11c0762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc11c103800 0x7fc11c075850 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1040b2880 0x7fc1040b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:55.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 --2- 172.21.15.47:0/3773041442 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc11c1041b0 0x7fc11c075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 >> 172.21.15.47:0/3773041442 conn(0x7fc11c07a6b0 msgr2=0x7fc11c1018b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 shutdown_connections 2024-05-11T12:23:55.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:55.054+0000 7fc122cf3700 1 -- 172.21.15.47:0/3773041442 wait complete. 2024-05-11T12:23:55.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:54 smithi050 ceph-mon[34072]: pgmap v379: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:54 smithi047 ceph-mon[30375]: pgmap v379: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:54 smithi064 ceph-mon[34537]: pgmap v379: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:55.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-11T12:21:13.603454+0000', 'modified': '2024-05-11T12:21:30.461773+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.50:6835/1314438872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6834', 'nonce': 1314438872}, {'type': 'v1', 'addr': '172.21.15.50:6835', 'nonce': 1314438872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.50:6837/3384616382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.50:6836', 'nonce': 3384616382}, {'type': 'v1', 'addr': '172.21.15.50:6837', 'nonce': 3384616382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24485, 'qdb_cluster': [24485]} 2024-05-11T12:23:55.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-11T12:23:55.740 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.h status 2024-05-11T12:23:55.994 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:23:56.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:55 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3773041442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:23:56.265 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:55 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3773041442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:23:56.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:55 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3773041442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:23:57.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:56 smithi047 ceph-mon[30375]: pgmap v380: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:57.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:56 smithi064 ceph-mon[34537]: pgmap v380: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:57.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:56 smithi050 ceph-mon[34072]: pgmap v380: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 -- 172.21.15.47:0/2381699137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce74109640 msgr2=0x7fce74109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 --2- 172.21.15.47:0/2381699137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce74109640 0x7fce74109a20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fce700099c0 tx=0x7fce70009cd0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 -- 172.21.15.47:0/2381699137 shutdown_connections 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 --2- 172.21.15.47:0/2381699137 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce7406c6e0 0x7fce74110410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 --2- 172.21.15.47:0/2381699137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce74109640 0x7fce74109a20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 --2- 172.21.15.47:0/2381699137 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7406bd20 0x7fce7406c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 -- 172.21.15.47:0/2381699137 >> 172.21.15.47:0/2381699137 conn(0x7fce740794f0 msgr2=0x7fce74079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 -- 172.21.15.47:0/2381699137 shutdown_connections 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.749+0000 7fce7b996700 1 -- 172.21.15.47:0/2381699137 wait complete. 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 Processor -- start 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 -- start start 2024-05-11T12:23:57.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7406bd20 0x7fce741024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce7406c6e0 0x7fce74102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce74109640 0x7fce74102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fce74113750 con 0x7fce7406c6e0 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce741135d0 con 0x7fce74109640 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.750+0000 7fce7b996700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce741138d0 con 0x7fce7406bd20 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce7406c6e0 0x7fce74102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce7406c6e0 0x7fce74102a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54208/0 (socket says 172.21.15.47:54208) 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 -- 172.21.15.47:0/2589957781 learned_addr learned my addr 172.21.15.47:0/2589957781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce79732700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7406bd20 0x7fce741024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce79f33700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce74109640 0x7fce74102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 -- 172.21.15.47:0/2589957781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7406bd20 msgr2=0x7fce741024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7406bd20 0x7fce741024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 -- 172.21.15.47:0/2589957781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce74109640 msgr2=0x7fce74102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce74109640 0x7fce74102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 -- 172.21.15.47:0/2589957781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce741a8050 con 0x7fce7406c6e0 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce78f31700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce7406c6e0 0x7fce74102a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c9b0 tx=0x7fce6400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce6a7fc700 1 -- 172.21.15.47:0/2589957781 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64007b00 con 0x7fce7406c6e0 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce741a8340 con 0x7fce7406c6e0 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce6a7fc700 1 -- 172.21.15.47:0/2589957781 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce64007ca0 con 0x7fce7406c6e0 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.751+0000 7fce6a7fc700 1 -- 172.21.15.47:0/2589957781 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64018660 con 0x7fce7406c6e0 2024-05-11T12:23:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.752+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a88d0 con 0x7fce7406c6e0 2024-05-11T12:23:57.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.754+0000 7fce6a7fc700 1 -- 172.21.15.47:0/2589957781 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce64004830 con 0x7fce7406c6e0 2024-05-11T12:23:57.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.755+0000 7fce6a7fc700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce600b2880 0x7fce600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.755+0000 7fce79732700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce600b2880 0x7fce600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.755+0000 7fce6a7fc700 1 -- 172.21.15.47:0/2589957781 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fce640e22f0 con 0x7fce7406c6e0 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.755+0000 7fce8622cb80 1 Processor -- start 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.755+0000 7fce79732700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce600b2880 0x7fce600b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce70009990 tx=0x7fce70009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.755+0000 7fce8622cb80 1 -- start start 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7ff1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7fec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7feca10 0x5591a7ff8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce8622cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5591a7f3e810 con 0x5591a7fec050 2024-05-11T12:23:57.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce8622cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5591a7edd770 con 0x5591a7ff1a80 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce8622cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5591a7f3a7d0 con 0x5591a7feca10 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7fec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7fec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54218/0 (socket says 172.21.15.47:54218) 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 -- 172.21.15.47:0/2586463264 learned_addr learned my addr 172.21.15.47:0/2586463264 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce79732700 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7ff1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce79f33700 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7feca10 0x5591a7ff8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 -- 172.21.15.47:0/2586463264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7feca10 msgr2=0x5591a7ff8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7feca10 0x5591a7ff8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 -- 172.21.15.47:0/2586463264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 msgr2=0x5591a7ff1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7ff1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 -- 172.21.15.47:0/2586463264 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a7e8b880 con 0x5591a7fec050 2024-05-11T12:23:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce79732700 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7ff1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce79f33700 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7feca10 0x5591a7ff8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce78f31700 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7fec4d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce6400afd0 tx=0x7fce640b2100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c4153f8390676a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce5f7fe700 1 -- 172.21.15.47:0/2586463264 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640a9430 con 0x5591a7fec050 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce5f7fe700 1 -- 172.21.15.47:0/2586463264 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce640adbf0 con 0x5591a7fec050 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.756+0000 7fce5f7fe700 1 -- 172.21.15.47:0/2586463264 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640b2410 con 0x5591a7fec050 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 -- 172.21.15.47:0/2586463264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 msgr2=0x5591a7fec4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7fec4d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce6400afd0 tx=0x7fce640b2100 comp rx=0 tx=0).stop 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 -- 172.21.15.47:0/2586463264 shutdown_connections 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7feca10 0x5591a7ff8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7ff1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2586463264 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7fec4d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 -- 172.21.15.47:0/2586463264 >> 172.21.15.47:0/2586463264 conn(0x5591a7f18e10 msgr2=0x5591a7fe6940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 -- 172.21.15.47:0/2586463264 shutdown_connections 2024-05-11T12:23:57.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.757+0000 7fce8622cb80 1 -- 172.21.15.47:0/2586463264 wait complete. 2024-05-11T12:23:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 -- start start 2024-05-11T12:23:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 Processor -- start 2024-05-11T12:23:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7ff8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7feca10 0x5591a7ff5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7ff1a80 0x5591a7ff5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5591a7ff8e40 con 0x5591a7fec050 2024-05-11T12:23:57.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5591a7ea4be0 con 0x5591a7feca10 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5591a7ff8fc0 con 0x5591a7ff1a80 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce8622cb80 10 client.0 _ll_register_callbacks cb 0x5591a7ed68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce79732700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7ff8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.758+0000 7fce79732700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7ff8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54230/0 (socket says 172.21.15.47:54230) 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79732700 1 -- 172.21.15.47:0/1640264423 learned_addr learned my addr 172.21.15.47:0/1640264423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79f33700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7ff1a80 0x5591a7ff5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce78f31700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7feca10 0x5591a7ff5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79732700 1 -- 172.21.15.47:0/1640264423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7ff1a80 msgr2=0x5591a7ff5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79732700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7ff1a80 0x5591a7ff5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79732700 1 -- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7feca10 msgr2=0x5591a7ff5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79732700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7feca10 0x5591a7ff5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79732700 1 -- 172.21.15.47:0/1640264423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a7eda640 con 0x5591a7fec050 2024-05-11T12:23:57.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce78f31700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7feca10 0x5591a7ff5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce79732700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7ff8590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce700098e0 tx=0x7fce70009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a89ede2009a28ac server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce5cff9700 1 -- 172.21.15.47:0/1640264423 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70034060 con 0x5591a7fec050 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce5cff9700 1 -- 172.21.15.47:0/1640264423 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fce7000f660 con 0x5591a7fec050 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce640adbf0 con 0x5591a7fec050 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce5cff9700 1 -- 172.21.15.47:0/1640264423 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce70024e20 con 0x5591a7fec050 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce8622cb80 10 client.15039 fetch_fsmap learned FSMap version 20 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.759+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5591a8003700 con 0x5591a7fec050 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce5cff9700 1 -- 172.21.15.47:0/1640264423 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce640adbf0 con 0x5591a7fec050 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce5cff9700 1 -- 172.21.15.47:0/1640264423 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fce7002fae0 con 0x5591a7fec050 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce8622cb80 10 client.15039 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce8622cb80 10 client.15039 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce8622cb80 20 client.15039 populate_metadata read hostname 'smithi047' 2024-05-11T12:23:57.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce8622cb80 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8005ce0 0x5591a8007150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce8622cb80 4 client.15039 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591a7f4c1f0 con 0x5591a8005ce0 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.760+0000 7fce78f31700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8005ce0 0x5591a8007150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.761+0000 7fce78f31700 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8005ce0 0x5591a8007150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d375e59275de58 server_cookie=3ebf7f28216d3edf in_seq=0 out_seq=0 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.761+0000 7fce5cff9700 10 client.15039 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.761+0000 7fce5cff9700 1 -- 172.21.15.47:0/1640264423 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5591a7f4c1f0 con 0x5591a8005ce0 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.761+0000 7fce5cff9700 10 client.15039 handle_command_reply: tid=0 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.761+0000 7fce8622cb80 1 client.15039 shutdown 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 msgr2=0x5591a7ff8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7ff8590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce700098e0 tx=0x7fce70009310 comp rx=0 tx=0).stop 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 shutdown_connections 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8005ce0 0x5591a8007150 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7ff1a80 0x5591a7ff5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7fec050 0x5591a7ff8590 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 --2- 172.21.15.47:0/1640264423 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7feca10 0x5591a7ff5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 >> 172.21.15.47:0/1640264423 conn(0x5591a7f18e10 msgr2=0x5591a7fe1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:57.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce5cff9700 0 client.15039 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:57.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 shutdown_connections 2024-05-11T12:23:57.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 1 -- 172.21.15.47:0/1640264423 wait complete. 2024-05-11T12:23:57.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.762+0000 7fce8622cb80 20 client.15039 trim_cache size 0 max 16384 2024-05-11T12:23:57.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 Processor -- start 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 -- start start 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a7ff1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a7fe0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7fe10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5591a8005490 con 0x5591a7feca10 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5591a8005b60 con 0x5591a7ff1a80 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce8622cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5591a8006230 con 0x5591a7fec050 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a7fe0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a7fe0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54240/0 (socket says 172.21.15.47:54240) 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce78f31700 1 -- 172.21.15.47:0/75349731 learned_addr learned my addr 172.21.15.47:0/75349731 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce79732700 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a7ff1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce79f33700 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7fe10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.783+0000 7fce78f31700 1 -- 172.21.15.47:0/75349731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 msgr2=0x5591a7ff1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce78f31700 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a7ff1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce78f31700 1 -- 172.21.15.47:0/75349731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 msgr2=0x5591a7fe10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce78f31700 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7fe10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce78f31700 1 -- 172.21.15.47:0/75349731 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a7fe3160 con 0x5591a7feca10 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce79f33700 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7fe10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce78f31700 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a7fe0b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fce64004e20 tx=0x7fce640a5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debad5affd5e49d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce5effd700 1 -- 172.21.15.47:0/75349731 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640dfd70 con 0x5591a7feca10 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce5effd700 1 -- 172.21.15.47:0/75349731 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce640a5bb0 con 0x5591a7feca10 2024-05-11T12:23:57.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.784+0000 7fce5effd700 1 -- 172.21.15.47:0/75349731 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640aadd0 con 0x5591a7feca10 2024-05-11T12:23:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 -- 172.21.15.47:0/75349731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 msgr2=0x5591a7fe0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a7fe0b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fce64004e20 tx=0x7fce640a5040 comp rx=0 tx=0).stop 2024-05-11T12:23:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 -- 172.21.15.47:0/75349731 shutdown_connections 2024-05-11T12:23:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a7fe10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a7ff1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 --2- 172.21.15.47:0/75349731 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a7fe0b80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 -- 172.21.15.47:0/75349731 >> 172.21.15.47:0/75349731 conn(0x5591a7f18e10 msgr2=0x5591a7ff0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 -- 172.21.15.47:0/75349731 shutdown_connections 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 -- 172.21.15.47:0/75349731 wait complete. 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 -- start start 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 Processor -- start 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.785+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a80105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a8010b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce8622cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a8009570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce8622cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5591a8006fd0 con 0x5591a7feca10 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce8622cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5591a8006900 con 0x5591a7ff1a80 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce8622cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5591a80076a0 con 0x5591a7fec050 2024-05-11T12:23:57.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce8622cb80 10 client.0 _ll_register_callbacks cb 0x5591a7ed68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a8010b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce79732700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a80105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a8010b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54246/0 (socket says 172.21.15.47:54246) 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 -- 172.21.15.47:0/2081161162 learned_addr learned my addr 172.21.15.47:0/2081161162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce79732700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a80105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:58638/0 (socket says 172.21.15.47:58638) 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce79f33700 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a8009570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 -- 172.21.15.47:0/2081161162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 msgr2=0x5591a80105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a80105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 -- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 msgr2=0x5591a8009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a8009570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 -- 172.21.15.47:0/2081161162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a8009ab0 con 0x5591a7feca10 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce79732700 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a80105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce78f31700 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a8010b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fce640074a0 tx=0x7fce64007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af5e3971e8cbaca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:23:57.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce5cff9700 1 -- 172.21.15.47:0/2081161162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce640a5f10 con 0x5591a7feca10 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce5cff9700 1 -- 172.21.15.47:0/2081161162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fce640dfd70 con 0x5591a7feca10 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.786+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce640a5bb0 con 0x5591a7feca10 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce5cff9700 1 -- 172.21.15.47:0/2081161162 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce640b2210 con 0x5591a7feca10 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 10 client.15045 fetch_fsmap learned FSMap version 20 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5591a8071930 con 0x5591a7feca10 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce5cff9700 1 -- 172.21.15.47:0/2081161162 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce640a5bb0 con 0x5591a7feca10 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce5cff9700 1 -- 172.21.15.47:0/2081161162 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fce6406f040 con 0x5591a7feca10 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 10 client.15045 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 20 client.15045 populate_metadata read hostname 'smithi047' 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8075ce0 0x5591a80749f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 4 client.15045 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-11T12:23:57.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "status"}) -- 0x5591a7f46290 con 0x5591a8075ce0 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.787+0000 7fce79732700 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8075ce0 0x5591a80749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce79732700 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8075ce0 0x5591a80749f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d02d2afd4a36379 server_cookie=bfd6d60ae3a45226 in_seq=0 out_seq=0 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce5cff9700 10 client.15045 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce5cff9700 1 -- 172.21.15.47:0/2081161162 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5591a7f46290 con 0x5591a8075ce0 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce5cff9700 10 client.15045 handle_command_reply: tid=0 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 client.15045 shutdown 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 msgr2=0x5591a8010b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a8010b10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fce640074a0 tx=0x7fce64007f10 comp rx=0 tx=0).stop 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 shutdown_connections 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5591a8075ce0 0x5591a80749f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce5cff9700 0 client.15045 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5591a7ff1a80 0x5591a8009570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5591a7fec050 0x5591a80105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 --2- 172.21.15.47:0/2081161162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5591a7feca10 0x5591a8010b10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 >> 172.21.15.47:0/2081161162 conn(0x5591a7f18e10 msgr2=0x5591a7feb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 shutdown_connections 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 1 -- 172.21.15.47:0/2081161162 wait complete. 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.788+0000 7fce8622cb80 20 client.15045 trim_cache size 0 max 16384 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "9b1a9a48-0f8f-11ef-bc98-c7b262605968", 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 24485, 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-05-11T12:23:57.794 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-05-11T12:23:57.795 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 163.196877543, 2024-05-11T12:23:57.795 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 20, 2024-05-11T12:23:57.795 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 83, 2024-05-11T12:23:57.795 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 79, 2024-05-11T12:23:57.795 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 194.32067084299999 2024-05-11T12:23:57.795 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce600b2880 msgr2=0x7fce600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce600b2880 0x7fce600b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fce70009990 tx=0x7fce70009650 comp rx=0 tx=0).stop 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce7406c6e0 msgr2=0x7fce74102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce7406c6e0 0x7fce74102a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fce6400c9b0 tx=0x7fce6400ccc0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 shutdown_connections 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce600b2880 0x7fce600b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce74109640 0x7fce74102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce7406bd20 0x7fce741024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 --2- 172.21.15.47:0/2589957781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce7406c6e0 0x7fce74102a00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:23:57.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 >> 172.21.15.47:0/2589957781 conn(0x7fce740794f0 msgr2=0x7fce74100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:23:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 shutdown_connections 2024-05-11T12:23:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:23:57.789+0000 7fce7b996700 1 -- 172.21.15.47:0/2589957781 wait complete. 2024-05-11T12:23:58.625 INFO:teuthology.run_tasks:Running task sequential... 2024-05-11T12:23:58.989 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-11T12:23:58.989 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-11T12:23:58.989 INFO:teuthology.task.print:Enabling mgr modules 2024-05-11T12:23:58.990 INFO:teuthology.run_tasks:Running task kclient... 2024-05-11T12:23:59.084 INFO:tasks.kclient:Mounting kernel clients... 2024-05-11T12:23:59.085 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-11T12:23:59.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:23:59.085 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-05-11T12:23:59.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:23:59.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-05-11T12:23:59.183 INFO:teuthology.orchestra.run.smithi047.stderr:Cannot find device "ceph-brx" 2024-05-11T12:23:59.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T12:23:59.185 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-11T12:23:59.185 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-11T12:23:59.185 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:23:59.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs ls 2024-05-11T12:23:59.253 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:23:58 smithi047 ceph-mon[30375]: pgmap v381: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:59.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:23:58 smithi064 ceph-mon[34537]: pgmap v381: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:59.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:23:58 smithi050 ceph-mon[34072]: pgmap v381: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:23:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:01.178 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:00 smithi047 ceph-mon[30375]: pgmap v382: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 -- 172.21.15.47:0/2716841664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f581035d0 msgr2=0x7f0f58103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/2716841664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f581035d0 0x7f0f58103a50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009990 tx=0x7f0f4c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 -- 172.21.15.47:0/2716841664 shutdown_connections 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/2716841664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f58103f90 0x7f0f58110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/2716841664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58109650 0x7f0f58109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/2716841664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f581035d0 0x7f0f58103a50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 -- 172.21.15.47:0/2716841664 >> 172.21.15.47:0/2716841664 conn(0x7f0f5807a700 msgr2=0x7f0f5807ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 -- 172.21.15.47:0/2716841664 shutdown_connections 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.285+0000 7f0f5f89b700 1 -- 172.21.15.47:0/2716841664 wait complete. 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 Processor -- start 2024-05-11T12:24:01.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 -- start start 2024-05-11T12:24:01.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f581035d0 0x7f0f5819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:01.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58103f90 0x7f0f5819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:01.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f58109650 0x7f0f58199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f581137c0 con 0x7f0f58109650 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58113640 con 0x7f0f58103f90 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5f89b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58113940 con 0x7f0f581035d0 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.286+0000 7f0f5de38700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f58109650 0x7f0f58199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f58109650 0x7f0f58199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54270/0 (socket says 172.21.15.47:54270) 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 -- 172.21.15.47:0/1047732587 learned_addr learned my addr 172.21.15.47:0/1047732587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5d637700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f581035d0 0x7f0f5819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5ce36700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58103f90 0x7f0f5819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 -- 172.21.15.47:0/1047732587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f581035d0 msgr2=0x7f0f5819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f581035d0 0x7f0f5819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 -- 172.21.15.47:0/1047732587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58103f90 msgr2=0x7f0f5819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58103f90 0x7f0f5819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 -- 172.21.15.47:0/1047732587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5819a180 con 0x7f0f58109650 2024-05-11T12:24:01.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5de38700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f58109650 0x7f0f58199a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000d980 tx=0x7f0f5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:01.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f567fc700 1 -- 172.21.15.47:0/1047732587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f50017070 con 0x7f0f58109650 2024-05-11T12:24:01.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f567fc700 1 -- 172.21.15.47:0/1047732587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f50005020 con 0x7f0f58109650 2024-05-11T12:24:01.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5819a470 con 0x7f0f58109650 2024-05-11T12:24:01.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f567fc700 1 -- 172.21.15.47:0/1047732587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f500099b0 con 0x7f0f58109650 2024-05-11T12:24:01.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.287+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581a6730 con 0x7f0f58109650 2024-05-11T12:24:01.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.289+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5806a240 con 0x7f0f58109650 2024-05-11T12:24:01.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.290+0000 7f0f567fc700 1 -- 172.21.15.47:0/1047732587 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0f50004a30 con 0x7f0f58109650 2024-05-11T12:24:01.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.291+0000 7f0f567fc700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f400b2900 0x7f0f400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:01.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.291+0000 7f0f5d637700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f400b2900 0x7f0f400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:01.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.291+0000 7f0f567fc700 1 -- 172.21.15.47:0/1047732587 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0f500e2530 con 0x7f0f58109650 2024-05-11T12:24:01.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.292+0000 7f0f5d637700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f400b2900 0x7f0f400b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0f5819ab60 tx=0x7f0f48007a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:01.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.294+0000 7f0f567fc700 1 -- 172.21.15.47:0/1047732587 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f50020020 con 0x7f0f58109650 2024-05-11T12:24:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:00 smithi064 ceph-mon[34537]: pgmap v382: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:01.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:00 smithi050 ceph-mon[34072]: pgmap v382: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.558+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0f5819af80 con 0x7f0f58109650 2024-05-11T12:24:01.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.558+0000 7f0f567fc700 1 -- 172.21.15.47:0/1047732587 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f0f5819af80 con 0x7f0f58109650 2024-05-11T12:24:01.558 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-11T12:24:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f400b2900 msgr2=0x7f0f400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f400b2900 0x7f0f400b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0f5819ab60 tx=0x7f0f48007a30 comp rx=0 tx=0).stop 2024-05-11T12:24:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f58109650 msgr2=0x7f0f58199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f58109650 0x7f0f58199a20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000d980 tx=0x7f0f5000dc90 comp rx=0 tx=0).stop 2024-05-11T12:24:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 shutdown_connections 2024-05-11T12:24:01.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f58109650 0x7f0f58199a20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0f581035d0 0x7f0f5819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0f400b2900 0x7f0f400b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 --2- 172.21.15.47:0/1047732587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0f58103f90 0x7f0f5819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.560+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 >> 172.21.15.47:0/1047732587 conn(0x7f0f5807a700 msgr2=0x7f0f581021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.561+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 shutdown_connections 2024-05-11T12:24:01.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:01.561+0000 7f0f5f89b700 1 -- 172.21.15.47:0/1047732587 wait complete. 2024-05-11T12:24:02.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:01 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1047732587' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T12:24:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:01 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1047732587' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-11T12:24:02.389 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:24:02.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:24:02.390 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:24:02.447 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-05-11T12:24:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-05-11T12:24:02.448 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-11T12:24:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-05-11T12:24:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-05-11T12:24:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-11T12:24:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42030sec preferred_lft 42030sec 2024-05-11T12:24:02.448 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-05-11T12:24:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-05-11T12:24:02.449 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-11T12:24:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-05-11T12:24:02.449 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-05-11T12:24:02.449 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-11T12:24:02.449 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:24:02.449 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-05-11T12:24:02.449 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-05-11T12:24:02.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-05-11T12:24:02.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-05-11T12:24:02.450 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-11T12:24:02.450 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-05-11T12:24:02.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:01 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1047732587' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-11T12:24:02.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:24:02.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:02.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:24:02.674 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-11T12:24:02.701 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-05-11T12:24:02.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:24:02.702 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-05-11T12:24:02.766 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-05-11T12:24:02.767 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-11T12:24:02.767 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-11T12:24:02.767 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-11T12:24:02.767 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-11T12:24:02.767 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:24:02.767 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-05-11T12:24:02.767 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-11T12:24:02.768 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-11T12:24:02.768 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-11T12:24:02.768 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-05-11T12:24:02.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:24:03.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:03.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:24:03.077 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-05-11T12:24:03.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:24:03.094 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-05-11T12:24:03.150 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:24:03.150 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-05-11T12:24:03.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:24:03.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-11T12:24:03.150 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-05-11T12:24:03.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:24:03.248 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:02 smithi047 ceph-mon[30375]: pgmap v383: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:24:03.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:03.271 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-11T12:24:03.272 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-05-11T12:24:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:24:03.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:02 smithi064 ceph-mon[34537]: pgmap v383: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:24:03.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:03.409 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-11T12:24:03.409 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-05-11T12:24:03.409 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-05-11T12:24:03.409 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-05-11T12:24:03.409 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-05-11T12:24:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:02 smithi050 ceph-mon[34072]: pgmap v383: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:24:03.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:24:03.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:03.491 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:24:03.491 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-11T12:24:03.506 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-11T12:24:03.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-11T12:24:03.507 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:24:03.562 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-11T12:24:03.562 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-11T12:24:03.562 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/e44abbf6-31a0-46a6-8414-5003f221d662 2024-05-11T12:24:03.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:24:03.563 DEBUG:teuthology.orchestra.run.smithi047:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/e44abbf6-31a0-46a6-8414-5003f221d662 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-05-11T12:24:03.655 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-05-11T12:24:03.655 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-05-11T12:24:03.655 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-05-11T12:24:03.793 INFO:teuthology.orchestra.run.smithi047.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-05-11T12:24:03.793 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-05-11T12:24:03.793 INFO:teuthology.orchestra.run.smithi047.stdout:invalid new device string format 2024-05-11T12:24:03.793 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: resolved to: "172.21.15.47:3300,172.21.15.50:3300,172.21.15.64:3300" 2024-05-11T12:24:03.793 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.47:3300,172.21.15.50:3300,172.21.15.64:3300:/volumes/qa/sv_1/e44abbf6-31a0-46a6-8414-5003f221d662 2024-05-11T12:24:03.793 INFO:teuthology.orchestra.run.smithi047.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=9b1a9a48-0f8f-11ef-bc98-c7b262605968" will pass to kernel 2024-05-11T12:24:03.794 INFO:teuthology.orchestra.run.smithi047.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-11T12:24:03.794 INFO:teuthology.orchestra.run.smithi047.stdout: You just mounted a file system that supports labels which does not 2024-05-11T12:24:03.794 INFO:teuthology.orchestra.run.smithi047.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-11T12:24:03.794 INFO:teuthology.orchestra.run.smithi047.stdout: applications will generate AVC messages and not be allowed access to 2024-05-11T12:24:03.794 INFO:teuthology.orchestra.run.smithi047.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-11T12:24:03.796 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-11T12:24:03.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:24:03.796 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-11T12:24:03.826 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:24:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:24:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:03.925 INFO:teuthology.orchestra.run.smithi047.stdout:client24769 2024-05-11T12:24:03.926 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:24:03.926 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:24:03.943 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:24:04.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:24:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:04.042 INFO:teuthology.orchestra.run.smithi047.stdout:client24769 2024-05-11T12:24:04.043 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-11T12:24:04.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-11T12:24:04.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:04.101 INFO:teuthology.orchestra.run.smithi047.stdout:client24769 2024-05-11T12:24:04.102 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968.client24769/status') 2024-05-11T12:24:04.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-11T12:24:04.162 INFO:teuthology.orchestra.run.smithi047.stdout:instance: client.24769 (3)172.21.15.47:0/895451146 2024-05-11T12:24:04.162 INFO:teuthology.orchestra.run.smithi047.stdout:blocklisted: false 2024-05-11T12:24:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records in 2024-05-11T12:24:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:0+1 records out 2024-05-11T12:24:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:70 bytes copied, 2.4288e-05 s, 2.9 MB/s 2024-05-11T12:24:04.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-11T12:24:04.170 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-11T12:24:04.184 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-11T12:24:04.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:24:04.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 -- 172.21.15.47:0/3723495261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c1019b0 msgr2=0x7f0a2c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 --2- 172.21.15.47:0/3723495261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c1019b0 0x7f0a2c101dd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009ca0 tx=0x7f0a2802a8c0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 -- 172.21.15.47:0/3723495261 shutdown_connections 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 --2- 172.21.15.47:0/3723495261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a2c103560 0x7f0a2c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 --2- 172.21.15.47:0/3723495261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a2c102ba0 0x7f0a2c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 --2- 172.21.15.47:0/3723495261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c1019b0 0x7f0a2c101dd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 -- 172.21.15.47:0/3723495261 >> 172.21.15.47:0/3723495261 conn(0x7f0a2c0fcf20 msgr2=0x7f0a2c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 -- 172.21.15.47:0/3723495261 shutdown_connections 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.399+0000 7f0a34724640 1 -- 172.21.15.47:0/3723495261 wait complete. 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 Processor -- start 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 -- start start 2024-05-11T12:24:04.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a2c1019b0 0x7f0a2c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c102ba0 0x7f0a2c19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a2c103560 0x7f0a2c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10d5b0 con 0x7f0a2c102ba0 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10d430 con 0x7f0a2c103560 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a34724640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c10d730 con 0x7f0a2c1019b0 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c102ba0 0x7f0a2c19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c102ba0 0x7f0a2c19db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54284/0 (socket says 172.21.15.47:54284) 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 -- 172.21.15.47:0/2127948714 learned_addr learned my addr 172.21.15.47:0/2127948714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a324c0640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a2c1019b0 0x7f0a2c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a32cc1640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a2c103560 0x7f0a2c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 -- 172.21.15.47:0/2127948714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a2c1019b0 msgr2=0x7f0a2c06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a2c1019b0 0x7f0a2c06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 -- 172.21.15.47:0/2127948714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a2c103560 msgr2=0x7f0a2c1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a2c103560 0x7f0a2c1a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a31cbf640 1 -- 172.21.15.47:0/2127948714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c1a0a30 con 0x7f0a2c102ba0 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.400+0000 7f0a324c0640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a2c1019b0 0x7f0a2c06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.401+0000 7f0a31cbf640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c102ba0 0x7f0a2c19db30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b4a0 tx=0x7f0a1c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.401+0000 7f0a237fe640 1 -- 172.21.15.47:0/2127948714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c014070 con 0x7f0a2c102ba0 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.401+0000 7f0a237fe640 1 -- 172.21.15.47:0/2127948714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a1c010ac0 con 0x7f0a2c102ba0 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.401+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c1a0d20 con 0x7f0a2c102ba0 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.401+0000 7f0a237fe640 1 -- 172.21.15.47:0/2127948714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c017c30 con 0x7f0a2c102ba0 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.401+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c1a12e0 con 0x7f0a2c102ba0 2024-05-11T12:24:04.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.402+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c108da0 con 0x7f0a2c102ba0 2024-05-11T12:24:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.404+0000 7f0a237fe640 1 -- 172.21.15.47:0/2127948714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0a1c017070 con 0x7f0a2c102ba0 2024-05-11T12:24:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.405+0000 7f0a237fe640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a180b2870 0x7f0a180b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.405+0000 7f0a324c0640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a180b2870 0x7f0a180b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.405+0000 7f0a237fe640 1 -- 172.21.15.47:0/2127948714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0a1c0e1a60 con 0x7f0a2c102ba0 2024-05-11T12:24:04.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.405+0000 7f0a324c0640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a180b2870 0x7f0a180b4d30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a28000f80 tx=0x7f0a280093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:04.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.408+0000 7f0a237fe640 1 -- 172.21.15.47:0/2127948714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0a1c0a9650 con 0x7f0a2c102ba0 2024-05-11T12:24:04.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.603+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a2c02de00 con 0x7f0a2c102ba0 2024-05-11T12:24:04.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.604+0000 7f0a237fe640 1 -- 172.21.15.47:0/2127948714 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22917 (secure 0 0 0) 0x7f0a1c0a8f50 con 0x7f0a2c102ba0 2024-05-11T12:24:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:04.605 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:22:43.640272+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a180b2870 msgr2=0x7f0a180b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a180b2870 0x7f0a180b4d30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0a28000f80 tx=0x7f0a280093f0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c102ba0 msgr2=0x7f0a2c19db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c102ba0 0x7f0a2c19db30 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c00b4a0 tx=0x7f0a1c00b960 comp rx=0 tx=0).stop 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 shutdown_connections 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a180b2870 0x7f0a180b4d30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a2c103560 0x7f0a2c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a2c102ba0 0x7f0a2c19db30 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 --2- 172.21.15.47:0/2127948714 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a2c1019b0 0x7f0a2c06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 >> 172.21.15.47:0/2127948714 conn(0x7f0a2c0fcf20 msgr2=0x7f0a2c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 shutdown_connections 2024-05-11T12:24:04.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.607+0000 7f0a34724640 1 -- 172.21.15.47:0/2127948714 wait complete. 2024-05-11T12:24:04.618 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:16:11.517702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-11T12:21:06.576652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-11T12:21:10.443533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:24:04.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 -- 172.21.15.47:0/1077857719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 msgr2=0x7fa958109de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 --2- 172.21.15.47:0/1077857719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa958109de0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa950009ca0 tx=0x7fa95002a8c0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 -- 172.21.15.47:0/1077857719 shutdown_connections 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 --2- 172.21.15.47:0/1077857719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa958109de0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 --2- 172.21.15.47:0/1077857719 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958102b90 0x7fa958103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 --2- 172.21.15.47:0/1077857719 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9581019a0 0x7fa958101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 -- 172.21.15.47:0/1077857719 >> 172.21.15.47:0/1077857719 conn(0x7fa9580fcf30 msgr2=0x7fa9580ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 -- 172.21.15.47:0/1077857719 shutdown_connections 2024-05-11T12:24:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 -- 172.21.15.47:0/1077857719 wait complete. 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 Processor -- start 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.727+0000 7fa95da72640 1 -- start start 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa95da72640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9581019a0 0x7fa95810c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa95da72640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958102b90 0x7fa95810a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa95da72640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa95810aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa95da72640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa95810e5f0 con 0x7fa958103550 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa95da72640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa95810e470 con 0x7fa9581019a0 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa95da72640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa95810e770 con 0x7fa958102b90 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa95810aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9581019a0 0x7fa95810c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa95810aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54294/0 (socket says 172.21.15.47:54294) 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 -- 172.21.15.47:0/3150195469 learned_addr learned my addr 172.21.15.47:0/3150195469 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa94ffff640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958102b90 0x7fa95810a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 -- 172.21.15.47:0/3150195469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958102b90 msgr2=0x7fa95810a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958102b90 0x7fa95810a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 -- 172.21.15.47:0/3150195469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9581019a0 msgr2=0x7fa95810c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9581019a0 0x7fa95810c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 -- 172.21.15.47:0/3150195469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95810b730 con 0x7fa958103550 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa957fff640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa95810aea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa950031a20 tx=0x7fa950003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.728+0000 7fa9557fa640 1 -- 172.21.15.47:0/3150195469 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa950004240 con 0x7fa958103550 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.729+0000 7fa9557fa640 1 -- 172.21.15.47:0/3150195469 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa950003e10 con 0x7fa958103550 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.729+0000 7fa9557fa640 1 -- 172.21.15.47:0/3150195469 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9500305c0 con 0x7fa958103550 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.729+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9581a2dc0 con 0x7fa958103550 2024-05-11T12:24:04.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.729+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9581a3230 con 0x7fa958103550 2024-05-11T12:24:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.730+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa958108cc0 con 0x7fa958103550 2024-05-11T12:24:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.732+0000 7fa9557fa640 1 -- 172.21.15.47:0/3150195469 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa95002f660 con 0x7fa958103550 2024-05-11T12:24:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.733+0000 7fa9557fa640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa9440b2870 0x7fa9440b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:04.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.733+0000 7fa9577fe640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa9440b2870 0x7fa9440b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:04.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.733+0000 7fa9557fa640 1 -- 172.21.15.47:0/3150195469 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fa950103c40 con 0x7fa958103550 2024-05-11T12:24:04.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.733+0000 7fa9577fe640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa9440b2870 0x7fa9440b4d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9581029f0 tx=0x7fa94800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:04.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.736+0000 7fa9557fa640 1 -- 172.21.15.47:0/3150195469 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9500cb830 con 0x7fa958103550 2024-05-11T12:24:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.926+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa95802dd40 con 0x7fa958103550 2024-05-11T12:24:04.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.926+0000 7fa9557fa640 1 -- 172.21.15.47:0/3150195469 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fa9500cb130 con 0x7fa958103550 2024-05-11T12:24:04.926 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-05-11T12:24:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa9440b2870 msgr2=0x7fa9440b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa9440b2870 0x7fa9440b4d30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa9581029f0 tx=0x7fa94800a2d0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 msgr2=0x7fa95810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa95810aea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa950031a20 tx=0x7fa950003c20 comp rx=0 tx=0).stop 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 shutdown_connections 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa9440b2870 0x7fa9440b4d30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa958103550 0x7fa95810aea0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa958102b90 0x7fa95810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 --2- 172.21.15.47:0/3150195469 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa9581019a0 0x7fa95810c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 >> 172.21.15.47:0/3150195469 conn(0x7fa9580fcf30 msgr2=0x7fa958105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 shutdown_connections 2024-05-11T12:24:04.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:04.928+0000 7fa95da72640 1 -- 172.21.15.47:0/3150195469 wait complete. 2024-05-11T12:24:04.940 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-11T12:24:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 -- 172.21.15.47:0/2777524438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 msgr2=0x7efd7c1059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 --2- 172.21.15.47:0/2777524438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c1059c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efd6c009900 tx=0x7efd6c02a410 comp rx=0 tx=0).stop 2024-05-11T12:24:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 -- 172.21.15.47:0/2777524438 shutdown_connections 2024-05-11T12:24:05.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 --2- 172.21.15.47:0/2777524438 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd7c108830 0x7efd7c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 --2- 172.21.15.47:0/2777524438 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c105f00 0x7efd7c1082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 --2- 172.21.15.47:0/2777524438 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c1059c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 -- 172.21.15.47:0/2777524438 >> 172.21.15.47:0/2777524438 conn(0x7efd7c0fcf40 msgr2=0x7efd7c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 -- 172.21.15.47:0/2777524438 shutdown_connections 2024-05-11T12:24:05.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.048+0000 7efd80846640 1 -- 172.21.15.47:0/2777524438 wait complete. 2024-05-11T12:24:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 Processor -- start 2024-05-11T12:24:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 -- start start 2024-05-11T12:24:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd7c105f00 0x7efd7c101580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c108830 0x7efd7c101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10e5f0 con 0x7efd7c1035d0 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10e470 con 0x7efd7c108830 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd80846640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c10e770 con 0x7efd7c105f00 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd7a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd7a59c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54304/0 (socket says 172.21.15.47:54304) 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.049+0000 7efd7a59c640 1 -- 172.21.15.47:0/742893317 learned_addr learned my addr 172.21.15.47:0/742893317 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd79d9b640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd7c105f00 0x7efd7c101580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd7ad9d640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c108830 0x7efd7c101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd7a59c640 1 -- 172.21.15.47:0/742893317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd7c105f00 msgr2=0x7efd7c101580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd7a59c640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd7c105f00 0x7efd7c101580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd7a59c640 1 -- 172.21.15.47:0/742893317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c108830 msgr2=0x7efd7c101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd7a59c640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c108830 0x7efd7c101ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd7a59c640 1 -- 172.21.15.47:0/742893317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c102350 con 0x7efd7c1035d0 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd7a59c640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c102f40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd6c031500 tx=0x7efd6c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd6b7fe640 1 -- 172.21.15.47:0/742893317 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c03f030 con 0x7efd7c1035d0 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd6b7fe640 1 -- 172.21.15.47:0/742893317 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd6c003800 con 0x7efd7c1035d0 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c1a2d80 con 0x7efd7c1035d0 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd6b7fe640 1 -- 172.21.15.47:0/742893317 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c035070 con 0x7efd7c1035d0 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.050+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1a31f0 con 0x7efd7c1035d0 2024-05-11T12:24:05.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.051+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c0fea30 con 0x7efd7c1035d0 2024-05-11T12:24:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.053+0000 7efd6b7fe640 1 -- 172.21.15.47:0/742893317 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7efd6c0039b0 con 0x7efd7c1035d0 2024-05-11T12:24:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.054+0000 7efd6b7fe640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd640b2870 0x7efd640b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.054+0000 7efd79d9b640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd640b2870 0x7efd640b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.054+0000 7efd6b7fe640 1 -- 172.21.15.47:0/742893317 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7efd6c103370 con 0x7efd7c1035d0 2024-05-11T12:24:05.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.054+0000 7efd79d9b640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd640b2870 0x7efd640b4d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efd7c102cb0 tx=0x7efd70005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:05.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.057+0000 7efd6b7fe640 1 -- 172.21.15.47:0/742893317 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7efd6c0caf60 con 0x7efd7c1035d0 2024-05-11T12:24:05.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.248+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efd7c063760 con 0x7efd7c1035d0 2024-05-11T12:24:05.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.248+0000 7efd6b7fe640 1 -- 172.21.15.47:0/742893317 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7efd6c0ca860 con 0x7efd7c1035d0 2024-05-11T12:24:05.248 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-05-11T12:24:05.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd640b2870 msgr2=0x7efd640b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd640b2870 0x7efd640b4d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efd7c102cb0 tx=0x7efd70005cc0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 msgr2=0x7efd7c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c102f40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efd6c031500 tx=0x7efd6c0043c0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 shutdown_connections 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7efd640b2870 0x7efd640b4d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7efd7c108830 0x7efd7c101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7efd7c105f00 0x7efd7c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 --2- 172.21.15.47:0/742893317 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd7c1035d0 0x7efd7c102f40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.250+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 >> 172.21.15.47:0/742893317 conn(0x7efd7c0fcf40 msgr2=0x7efd7c103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.251+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 shutdown_connections 2024-05-11T12:24:05.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.251+0000 7efd80846640 1 -- 172.21.15.47:0/742893317 wait complete. 2024-05-11T12:24:05.260 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:04 smithi047 ceph-mon[30375]: pgmap v384: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:05.260 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:04 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2127948714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:24:05.260 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:04 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3150195469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:24:05.261 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-11T12:24:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:04 smithi064 ceph-mon[34537]: pgmap v384: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:04 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2127948714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:24:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:04 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3150195469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:24:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 -- 172.21.15.47:0/88589735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b0102ba0 msgr2=0x7f22b0103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 --2- 172.21.15.47:0/88589735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b0102ba0 0x7f22b0103020 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2298009900 tx=0x7f229802a410 comp rx=0 tx=0).stop 2024-05-11T12:24:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 -- 172.21.15.47:0/88589735 shutdown_connections 2024-05-11T12:24:05.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 --2- 172.21.15.47:0/88589735 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b0103560 0x7f22b0109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 --2- 172.21.15.47:0/88589735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b0102ba0 0x7f22b0103020 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 --2- 172.21.15.47:0/88589735 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b01019b0 0x7f22b0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 -- 172.21.15.47:0/88589735 >> 172.21.15.47:0/88589735 conn(0x7f22b00fcf40 msgr2=0x7f22b00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 -- 172.21.15.47:0/88589735 shutdown_connections 2024-05-11T12:24:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.369+0000 7f22b6904640 1 -- 172.21.15.47:0/88589735 wait complete. 2024-05-11T12:24:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 Processor -- start 2024-05-11T12:24:05.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 -- start start 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01019b0 0x7f22b0198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0102ba0 0x7f22b0199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b0103560 0x7f22b01a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010e600 con 0x7f22b01019b0 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010e480 con 0x7f22b0103560 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b6904640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f22b010e780 con 0x7f22b0102ba0 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22affff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01019b0 0x7f22b0198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22affff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01019b0 0x7f22b0198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:54324/0 (socket says 172.21.15.47:54324) 2024-05-11T12:24:05.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22affff640 1 -- 172.21.15.47:0/3177891403 learned_addr learned my addr 172.21.15.47:0/3177891403 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22b4ea1640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b0103560 0x7f22b01a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22affff640 1 -- 172.21.15.47:0/3177891403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0102ba0 msgr2=0x7f22b0199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22a7fff640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0102ba0 0x7f22b0199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22affff640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0102ba0 0x7f22b0199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.370+0000 7f22affff640 1 -- 172.21.15.47:0/3177891403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b0103560 msgr2=0x7f22b01a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22affff640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b0103560 0x7f22b01a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22affff640 1 -- 172.21.15.47:0/3177891403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22b01a0960 con 0x7f22b01019b0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22b4ea1640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b0103560 0x7f22b01a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22affff640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01019b0 0x7f22b0198d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d950 tx=0x7f22a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22adffb640 1 -- 172.21.15.47:0/3177891403 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a0017070 con 0x7f22b01019b0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22adffb640 1 -- 172.21.15.47:0/3177891403 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22a0009c30 con 0x7f22b01019b0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22b01a0c50 con 0x7f22b01019b0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22adffb640 1 -- 172.21.15.47:0/3177891403 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a000f620 con 0x7f22b01019b0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.371+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22b01a11a0 con 0x7f22b01019b0 2024-05-11T12:24:05.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.372+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22b0108d70 con 0x7f22b01019b0 2024-05-11T12:24:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.373+0000 7f22adffb640 1 -- 172.21.15.47:0/3177891403 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f22a001a070 con 0x7f22b01019b0 2024-05-11T12:24:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.374+0000 7f22adffb640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f229c0b2870 0x7f229c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.375+0000 7f22a7fff640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f229c0b2870 0x7f229c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:05.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.375+0000 7f22adffb640 1 -- 172.21.15.47:0/3177891403 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f22a00e1d90 con 0x7f22b01019b0 2024-05-11T12:24:05.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.375+0000 7f22a7fff640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f229c0b2870 0x7f229c0b4d30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f229802a930 tx=0x7f2298000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:05.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.378+0000 7f22adffb640 1 -- 172.21.15.47:0/3177891403 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22a0009de0 con 0x7f22b01019b0 2024-05-11T12:24:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:04 smithi050 ceph-mon[34072]: pgmap v384: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:04 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2127948714' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:24:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:04 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3150195469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:24:05.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.570+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22b002dcd0 con 0x7f22b01019b0 2024-05-11T12:24:05.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.571+0000 7f22adffb640 1 -- 172.21.15.47:0/3177891403 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f22a00a9a40 con 0x7f22b01019b0 2024-05-11T12:24:05.571 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f229c0b2870 msgr2=0x7f229c0b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f229c0b2870 0x7f229c0b4d30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f229802a930 tx=0x7f2298000bc0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01019b0 msgr2=0x7f22b0198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01019b0 0x7f22b0198d40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f22a000d950 tx=0x7f22a000de10 comp rx=0 tx=0).stop 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 shutdown_connections 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f229c0b2870 0x7f229c0b4d30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f22b0103560 0x7f22b01a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f22b0102ba0 0x7f22b0199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 --2- 172.21.15.47:0/3177891403 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f22b01019b0 0x7f22b0198d40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 >> 172.21.15.47:0/3177891403 conn(0x7f22b00fcf40 msgr2=0x7f22b0105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 shutdown_connections 2024-05-11T12:24:05.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:05.573+0000 7f22b6904640 1 -- 172.21.15.47:0/3177891403 wait complete. 2024-05-11T12:24:05.584 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-11T12:24:05.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:24:05.836 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:06.240 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:05 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/742893317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T12:24:06.240 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:05 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3177891403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T12:24:06.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:05 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/742893317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T12:24:06.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:05 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3177891403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T12:24:06.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:05 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/742893317' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T12:24:06.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:05 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3177891403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T12:24:07.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:06 smithi047 ceph-mon[30375]: pgmap v385: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-11T12:24:07.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:06 smithi064 ceph-mon[34537]: pgmap v385: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-11T12:24:07.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:06 smithi050 ceph-mon[34072]: pgmap v385: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 -- 172.21.15.47:0/2624478877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be006b230 msgr2=0x7f5be006b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 --2- 172.21.15.47:0/2624478877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be006b230 0x7f5be006b6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009990 tx=0x7f5bc8009ca0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 -- 172.21.15.47:0/2624478877 shutdown_connections 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 --2- 172.21.15.47:0/2624478877 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be006bbf0 0x7f5be01103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 --2- 172.21.15.47:0/2624478877 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5be01095c0 0x7f5be01099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 --2- 172.21.15.47:0/2624478877 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be006b230 0x7f5be006b6b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 -- 172.21.15.47:0/2624478877 >> 172.21.15.47:0/2624478877 conn(0x7f5be0079400 msgr2=0x7f5be0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 -- 172.21.15.47:0/2624478877 shutdown_connections 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.674+0000 7f5be5af9700 1 -- 172.21.15.47:0/2624478877 wait complete. 2024-05-11T12:24:07.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 Processor -- start 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 -- start start 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be006b230 0x7f5be019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5be006bbf0 0x7f5be019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be01095c0 0x7f5be0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5be0113750 con 0x7f5be01095c0 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01135d0 con 0x7f5be006b230 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5be5af9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01138d0 con 0x7f5be006bbf0 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.675+0000 7f5bdffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be01095c0 0x7f5be0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be01095c0 0x7f5be0199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52380/0 (socket says 172.21.15.47:52380) 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 -- 172.21.15.47:0/821155041 learned_addr learned my addr 172.21.15.47:0/821155041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bd6ffd700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5be006bbf0 0x7f5be019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdf7fe700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be006b230 0x7f5be019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:07.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 -- 172.21.15.47:0/821155041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5be006bbf0 msgr2=0x7f5be019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5be006bbf0 0x7f5be019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 -- 172.21.15.47:0/821155041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be006b230 msgr2=0x7f5be019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be006b230 0x7f5be019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 -- 172.21.15.47:0/821155041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be019a1d0 con 0x7f5be01095c0 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdffff700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be01095c0 0x7f5be0199a70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b550 tx=0x7f5bd800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdd7fa700 1 -- 172.21.15.47:0/821155041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8015020 con 0x7f5be01095c0 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be019a4c0 con 0x7f5be01095c0 2024-05-11T12:24:07.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdd7fa700 1 -- 172.21.15.47:0/821155041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bd8011ce0 con 0x7f5be01095c0 2024-05-11T12:24:07.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5bdd7fa700 1 -- 172.21.15.47:0/821155041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8010c90 con 0x7f5be01095c0 2024-05-11T12:24:07.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.676+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be01a6780 con 0x7f5be01095c0 2024-05-11T12:24:07.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.677+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be006a240 con 0x7f5be01095c0 2024-05-11T12:24:07.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.679+0000 7f5bdd7fa700 1 -- 172.21.15.47:0/821155041 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5bd8018020 con 0x7f5be01095c0 2024-05-11T12:24:07.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.680+0000 7f5bdd7fa700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5bcc0b2880 0x7f5bcc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:07.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.680+0000 7f5bdf7fe700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5bcc0b2880 0x7f5bcc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:07.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.680+0000 7f5bdd7fa700 1 -- 172.21.15.47:0/821155041 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f5bd80e1f00 con 0x7f5be01095c0 2024-05-11T12:24:07.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.680+0000 7f5bdf7fe700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5bcc0b2880 0x7f5bcc0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00060b0 tx=0x7f5bd000c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:07.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.683+0000 7f5bdd7fa700 1 -- 172.21.15.47:0/821155041 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5bd801f020 con 0x7f5be01095c0 2024-05-11T12:24:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.946+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5be019afa0 con 0x7f5be01095c0 2024-05-11T12:24:07.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.947+0000 7f5bdd7fa700 1 -- 172.21.15.47:0/821155041 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f5bd80a9b50 con 0x7f5be01095c0 2024-05-11T12:24:07.948 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:07.948 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:24:07.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5bcc0b2880 msgr2=0x7f5bcc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5bcc0b2880 0x7f5bcc0b4d40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bd00060b0 tx=0x7f5bd000c040 comp rx=0 tx=0).stop 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be01095c0 msgr2=0x7f5be0199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be01095c0 0x7f5be0199a70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800b550 tx=0x7f5bd800b860 comp rx=0 tx=0).stop 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 shutdown_connections 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5be01095c0 0x7f5be0199a70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5bcc0b2880 0x7f5bcc0b4d40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5be006b230 0x7f5be019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 --2- 172.21.15.47:0/821155041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5be006bbf0 0x7f5be019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 >> 172.21.15.47:0/821155041 conn(0x7f5be0079400 msgr2=0x7f5be0101270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 shutdown_connections 2024-05-11T12:24:07.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:07.950+0000 7f5be5af9700 1 -- 172.21.15.47:0/821155041 wait complete. 2024-05-11T12:24:07.952 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:24:08.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:07 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/821155041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:07 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/821155041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:08.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:07 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/821155041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:08.850 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-11T12:24:08.850 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-11T12:24:08.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 -- 172.21.15.47:0/1754345262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c102ba0 msgr2=0x7f411c103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 --2- 172.21.15.47:0/1754345262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c102ba0 0x7f411c103020 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4110009900 tx=0x7f411002a410 comp rx=0 tx=0).stop 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 -- 172.21.15.47:0/1754345262 shutdown_connections 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 --2- 172.21.15.47:0/1754345262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c103560 0x7f411c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 --2- 172.21.15.47:0/1754345262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c102ba0 0x7f411c103020 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 --2- 172.21.15.47:0/1754345262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411c1019b0 0x7f411c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 -- 172.21.15.47:0/1754345262 >> 172.21.15.47:0/1754345262 conn(0x7f411c0fcf20 msgr2=0x7f411c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 -- 172.21.15.47:0/1754345262 shutdown_connections 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.965+0000 7f4122464640 1 -- 172.21.15.47:0/1754345262 wait complete. 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 Processor -- start 2024-05-11T12:24:08.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 -- start start 2024-05-11T12:24:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411c1019b0 0x7f411c198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c102ba0 0x7f411c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c103560 0x7f411c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:08.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f411c10e600 con 0x7f411c103560 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f411c10e480 con 0x7f411c1019b0 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.966+0000 7f4122464640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f411c10e780 con 0x7f411c102ba0 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c103560 0x7f411c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c103560 0x7f411c1a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52394/0 (socket says 172.21.15.47:52394) 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 -- 172.21.15.47:0/3057220725 learned_addr learned my addr 172.21.15.47:0/3057220725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f411b7fe640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c102ba0 0x7f411c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f411bfff640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411c1019b0 0x7f411c198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 -- 172.21.15.47:0/3057220725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c102ba0 msgr2=0x7f411c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c102ba0 0x7f411c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 -- 172.21.15.47:0/3057220725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411c1019b0 msgr2=0x7f411c198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411c1019b0 0x7f411c198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 -- 172.21.15.47:0/3057220725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c1a0900 con 0x7f411c103560 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4120a01640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c103560 0x7f411c1a0260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f411400cad0 tx=0x7f411400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:08.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f41197fa640 1 -- 172.21.15.47:0/3057220725 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411400ae20 con 0x7f411c103560 2024-05-11T12:24:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f41197fa640 1 -- 172.21.15.47:0/3057220725 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4114005020 con 0x7f411c103560 2024-05-11T12:24:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c1a0bf0 con 0x7f411c103560 2024-05-11T12:24:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f41197fa640 1 -- 172.21.15.47:0/3057220725 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411400f6b0 con 0x7f411c103560 2024-05-11T12:24:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.967+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c1a10f0 con 0x7f411c103560 2024-05-11T12:24:08.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.968+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f411c108cd0 con 0x7f411c103560 2024-05-11T12:24:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.970+0000 7f41197fa640 1 -- 172.21.15.47:0/3057220725 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f41140051d0 con 0x7f411c103560 2024-05-11T12:24:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.971+0000 7f41197fa640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f41040b27f0 0x7f41040b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.971+0000 7f411bfff640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f41040b27f0 0x7f41040b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.971+0000 7f41197fa640 1 -- 172.21.15.47:0/3057220725 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f41140e2820 con 0x7f411c103560 2024-05-11T12:24:08.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.971+0000 7f411bfff640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f41040b27f0 0x7f41040b4cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f410c005db0 tx=0x7f410c005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:08.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:08.974+0000 7f41197fa640 1 -- 172.21.15.47:0/3057220725 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4114014030 con 0x7f411c103560 2024-05-11T12:24:09.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.169+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f411c062ea0 con 0x7f41040b27f0 2024-05-11T12:24:09.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.176+0000 7f41197fa640 1 -- 172.21.15.47:0/3057220725 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258997 (secure 0 0 0) 0x7f411c062ea0 con 0x7f41040b27f0 2024-05-11T12:24:09.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:09.181 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-05-11T12:24:09.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f41040b27f0 msgr2=0x7f41040b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f41040b27f0 0x7f41040b4cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f410c005db0 tx=0x7f410c005d40 comp rx=0 tx=0).stop 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c103560 msgr2=0x7f411c1a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c103560 0x7f411c1a0260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f411400cad0 tx=0x7f411400cf90 comp rx=0 tx=0).stop 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 shutdown_connections 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f41040b27f0 0x7f41040b4cb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f411c103560 0x7f411c1a0260 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f411c102ba0 0x7f411c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 --2- 172.21.15.47:0/3057220725 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f411c1019b0 0x7f411c198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 >> 172.21.15.47:0/3057220725 conn(0x7f411c0fcf20 msgr2=0x7f411c105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 shutdown_connections 2024-05-11T12:24:09.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.179+0000 7f4122464640 1 -- 172.21.15.47:0/3057220725 wait complete. 2024-05-11T12:24:09.190 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:08 smithi047 ceph-mon[30375]: pgmap v386: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-11T12:24:09.191 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-05-11T12:24:07.575917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653707,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":407,"num_read_kb":293,"num_write":217,"num_write_kb":2244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951874,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":670,"ondisk_log_size":670,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393340,"kb_used_data":10908,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124548484,"statfs":{"total":1151940427776,"available":1151537647616,"internally_reserved":0,"allocated":11169792,"data_stored":7304301,"data_compressed":246459,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002973"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553592+0000","last_change":"2024-05-11T12:21:11.536657+0000","last_active":"2024-05-11T12:23:11.553592+0000","last_peered":"2024-05-11T12:23:11.553592+0000","last_clean":"2024-05-11T12:23:11.553592+0000","last_became_active":"2024-05-11T12:21:11.536243+0000","last_became_peered":"2024-05-11T12:21:11.536243+0000","last_unstale":"2024-05-11T12:23:11.553592+0000","last_undegraded":"2024-05-11T12:23:11.553592+0000","last_fullsized":"2024-05-11T12:23:11.553592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:29:20.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545144+0000","last_change":"2024-05-11T12:21:11.527602+0000","last_active":"2024-05-11T12:23:11.545144+0000","last_peered":"2024-05-11T12:23:11.545144+0000","last_clean":"2024-05-11T12:23:11.545144+0000","last_became_active":"2024-05-11T12:21:11.527309+0000","last_became_peered":"2024-05-11T12:21:11.527309+0000","last_unstale":"2024-05-11T12:23:11.545144+0000","last_undegraded":"2024-05-11T12:23:11.545144+0000","last_fullsized":"2024-05-11T12:23:11.545144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:03.764570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556325+0000","last_change":"2024-05-11T12:21:11.529094+0000","last_active":"2024-05-11T12:23:11.556325+0000","last_peered":"2024-05-11T12:23:11.556325+0000","last_clean":"2024-05-11T12:23:11.556325+0000","last_became_active":"2024-05-11T12:21:11.528796+0000","last_became_peered":"2024-05-11T12:21:11.528796+0000","last_unstale":"2024-05-11T12:23:11.556325+0000","last_undegraded":"2024-05-11T12:23:11.556325+0000","last_fullsized":"2024-05-11T12:23:11.556325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:46:41.098066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561517+0000","last_change":"2024-05-11T12:21:11.528903+0000","last_active":"2024-05-11T12:23:11.561517+0000","last_peered":"2024-05-11T12:23:11.561517+0000","last_clean":"2024-05-11T12:23:11.561517+0000","last_became_active":"2024-05-11T12:21:11.528545+0000","last_became_peered":"2024-05-11T12:21:11.528545+0000","last_unstale":"2024-05-11T12:23:11.561517+0000","last_undegraded":"2024-05-11T12:23:11.561517+0000","last_fullsized":"2024-05-11T12:23:11.561517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:56.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544651+0000","last_change":"2024-05-11T12:21:11.521074+0000","last_active":"2024-05-11T12:23:11.544651+0000","last_peered":"2024-05-11T12:23:11.544651+0000","last_clean":"2024-05-11T12:23:11.544651+0000","last_became_active":"2024-05-11T12:21:11.520675+0000","last_became_peered":"2024-05-11T12:21:11.520675+0000","last_unstale":"2024-05-11T12:23:11.544651+0000","last_undegraded":"2024-05-11T12:23:11.544651+0000","last_fullsized":"2024-05-11T12:23:11.544651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:22:50.952687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547969+0000","last_change":"2024-05-11T12:21:42.671243+0000","last_active":"2024-05-11T12:23:11.547969+0000","last_peered":"2024-05-11T12:23:11.547969+0000","last_clean":"2024-05-11T12:23:11.547969+0000","last_became_active":"2024-05-11T12:21:42.670801+0000","last_became_peered":"2024-05-11T12:21:42.670801+0000","last_unstale":"2024-05-11T12:23:11.547969+0000","last_undegraded":"2024-05-11T12:23:11.547969+0000","last_fullsized":"2024-05-11T12:23:11.547969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:02.539855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540731+0000","last_change":"2024-05-11T12:22:44.095706+0000","last_active":"2024-05-11T12:23:11.540731+0000","last_peered":"2024-05-11T12:23:11.540731+0000","last_clean":"2024-05-11T12:23:11.540731+0000","last_became_active":"2024-05-11T12:22:44.095177+0000","last_became_peered":"2024-05-11T12:22:44.095177+0000","last_unstale":"2024-05-11T12:23:11.540731+0000","last_undegraded":"2024-05-11T12:23:11.540731+0000","last_fullsized":"2024-05-11T12:23:11.540731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:11.426607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545528+0000","last_change":"2024-05-11T12:21:11.532874+0000","last_active":"2024-05-11T12:23:11.545528+0000","last_peered":"2024-05-11T12:23:11.545528+0000","last_clean":"2024-05-11T12:23:11.545528+0000","last_became_active":"2024-05-11T12:21:11.532237+0000","last_became_peered":"2024-05-11T12:21:11.532237+0000","last_unstale":"2024-05-11T12:23:11.545528+0000","last_undegraded":"2024-05-11T12:23:11.545528+0000","last_fullsized":"2024-05-11T12:23:11.545528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:13.770719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.557701+0000","last_change":"2024-05-11T12:21:11.536465+0000","last_active":"2024-05-11T12:23:11.557701+0000","last_peered":"2024-05-11T12:23:11.557701+0000","last_clean":"2024-05-11T12:23:11.557701+0000","last_became_active":"2024-05-11T12:21:11.536057+0000","last_became_peered":"2024-05-11T12:21:11.536057+0000","last_unstale":"2024-05-11T12:23:11.557701+0000","last_undegraded":"2024-05-11T12:23:11.557701+0000","last_fullsized":"2024-05-11T12:23:11.557701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:22:45.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560592+0000","last_change":"2024-05-11T12:21:11.528187+0000","last_active":"2024-05-11T12:23:11.560592+0000","last_peered":"2024-05-11T12:23:11.560592+0000","last_clean":"2024-05-11T12:23:11.560592+0000","last_became_active":"2024-05-11T12:21:11.527780+0000","last_became_peered":"2024-05-11T12:21:11.527780+0000","last_unstale":"2024-05-11T12:23:11.560592+0000","last_undegraded":"2024-05-11T12:23:11.560592+0000","last_fullsized":"2024-05-11T12:23:11.560592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:23.484372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560911+0000","last_change":"2024-05-11T12:21:11.543388+0000","last_active":"2024-05-11T12:23:11.560911+0000","last_peered":"2024-05-11T12:23:11.560911+0000","last_clean":"2024-05-11T12:23:11.560911+0000","last_became_active":"2024-05-11T12:21:11.542984+0000","last_became_peered":"2024-05-11T12:21:11.542984+0000","last_unstale":"2024-05-11T12:23:11.560911+0000","last_undegraded":"2024-05-11T12:23:11.560911+0000","last_fullsized":"2024-05-11T12:23:11.560911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:52.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561371+0000","last_change":"2024-05-11T12:21:11.523355+0000","last_active":"2024-05-11T12:23:11.561371+0000","last_peered":"2024-05-11T12:23:11.561371+0000","last_clean":"2024-05-11T12:23:11.561371+0000","last_became_active":"2024-05-11T12:21:11.523007+0000","last_became_peered":"2024-05-11T12:21:11.523007+0000","last_unstale":"2024-05-11T12:23:11.561371+0000","last_undegraded":"2024-05-11T12:23:11.561371+0000","last_fullsized":"2024-05-11T12:23:11.561371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:57.197028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548605+0000","last_change":"2024-05-11T12:21:11.528072+0000","last_active":"2024-05-11T12:23:11.548605+0000","last_peered":"2024-05-11T12:23:11.548605+0000","last_clean":"2024-05-11T12:23:11.548605+0000","last_became_active":"2024-05-11T12:21:11.527478+0000","last_became_peered":"2024-05-11T12:21:11.527478+0000","last_unstale":"2024-05-11T12:23:11.548605+0000","last_undegraded":"2024-05-11T12:23:11.548605+0000","last_fullsized":"2024-05-11T12:23:11.548605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:57:29.001602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544905+0000","last_change":"2024-05-11T12:21:11.526862+0000","last_active":"2024-05-11T12:23:11.544905+0000","last_peered":"2024-05-11T12:23:11.544905+0000","last_clean":"2024-05-11T12:23:11.544905+0000","last_became_active":"2024-05-11T12:21:11.526553+0000","last_became_peered":"2024-05-11T12:21:11.526553+0000","last_unstale":"2024-05-11T12:23:11.544905+0000","last_undegraded":"2024-05-11T12:23:11.544905+0000","last_fullsized":"2024-05-11T12:23:11.544905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:48.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540462+0000","last_change":"2024-05-11T12:21:42.669393+0000","last_active":"2024-05-11T12:23:11.540462+0000","last_peered":"2024-05-11T12:23:11.540462+0000","last_clean":"2024-05-11T12:23:11.540462+0000","last_became_active":"2024-05-11T12:21:42.669000+0000","last_became_peered":"2024-05-11T12:21:42.669000+0000","last_unstale":"2024-05-11T12:23:11.540462+0000","last_undegraded":"2024-05-11T12:23:11.540462+0000","last_fullsized":"2024-05-11T12:23:11.540462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:29.439588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.563015+0000","last_change":"2024-05-11T12:21:11.540698+0000","last_active":"2024-05-11T12:23:11.563015+0000","last_peered":"2024-05-11T12:23:11.563015+0000","last_clean":"2024-05-11T12:23:11.563015+0000","last_became_active":"2024-05-11T12:21:11.539899+0000","last_became_peered":"2024-05-11T12:21:11.539899+0000","last_unstale":"2024-05-11T12:23:11.563015+0000","last_undegraded":"2024-05-11T12:23:11.563015+0000","last_fullsized":"2024-05-11T12:23:11.563015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:48.222664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553880+0000","last_change":"2024-05-11T12:21:11.537055+0000","last_active":"2024-05-11T12:23:11.553880+0000","last_peered":"2024-05-11T12:23:11.553880+0000","last_clean":"2024-05-11T12:23:11.553880+0000","last_became_active":"2024-05-11T12:21:11.536625+0000","last_became_peered":"2024-05-11T12:21:11.536625+0000","last_unstale":"2024-05-11T12:23:11.553880+0000","last_undegraded":"2024-05-11T12:23:11.553880+0000","last_fullsized":"2024-05-11T12:23:11.553880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:33.453467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548280+0000","last_change":"2024-05-11T12:21:11.530254+0000","last_active":"2024-05-11T12:23:11.548280+0000","last_peered":"2024-05-11T12:23:11.548280+0000","last_clean":"2024-05-11T12:23:11.548280+0000","last_became_active":"2024-05-11T12:21:11.529890+0000","last_became_peered":"2024-05-11T12:21:11.529890+0000","last_unstale":"2024-05-11T12:23:11.548280+0000","last_undegraded":"2024-05-11T12:23:11.548280+0000","last_fullsized":"2024-05-11T12:23:11.548280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:36.100931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.549156+0000","last_change":"2024-05-11T12:21:11.532893+0000","last_active":"2024-05-11T12:23:11.549156+0000","last_peered":"2024-05-11T12:23:11.549156+0000","last_clean":"2024-05-11T12:23:11.549156+0000","last_became_active":"2024-05-11T12:21:11.532539+0000","last_became_peered":"2024-05-11T12:21:11.532539+0000","last_unstale":"2024-05-11T12:23:11.549156+0000","last_undegraded":"2024-05-11T12:23:11.549156+0000","last_fullsized":"2024-05-11T12:23:11.549156+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:14:57.888102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'4","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647560+0000","last_change":"2024-05-11T12:21:08.510909+0000","last_active":"2024-05-11T12:22:43.647560+0000","last_peered":"2024-05-11T12:22:43.647560+0000","last_clean":"2024-05-11T12:22:43.647560+0000","last_became_active":"2024-05-11T12:21:08.509986+0000","last_became_peered":"2024-05-11T12:21:08.509986+0000","last_unstale":"2024-05-11T12:22:43.647560+0000","last_undegraded":"2024-05-11T12:22:43.647560+0000","last_fullsized":"2024-05-11T12:22:43.647560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:03:36.430087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547054+0000","last_change":"2024-05-11T12:21:11.532403+0000","last_active":"2024-05-11T12:23:11.547054+0000","last_peered":"2024-05-11T12:23:11.547054+0000","last_clean":"2024-05-11T12:23:11.547054+0000","last_became_active":"2024-05-11T12:21:11.531907+0000","last_became_peered":"2024-05-11T12:21:11.531907+0000","last_unstale":"2024-05-11T12:23:11.547054+0000","last_undegraded":"2024-05-11T12:23:11.547054+0000","last_fullsized":"2024-05-11T12:23:11.547054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:53:27.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790479+0000","last_change":"2024-05-11T12:21:08.531664+0000","last_active":"2024-05-11T12:22:43.790479+0000","last_peered":"2024-05-11T12:22:43.790479+0000","last_clean":"2024-05-11T12:22:43.790479+0000","last_became_active":"2024-05-11T12:21:08.531319+0000","last_became_peered":"2024-05-11T12:21:08.531319+0000","last_unstale":"2024-05-11T12:22:43.790479+0000","last_undegraded":"2024-05-11T12:22:43.790479+0000","last_fullsized":"2024-05-11T12:22:43.790479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:16:08.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556428+0000","last_change":"2024-05-11T12:21:11.532132+0000","last_active":"2024-05-11T12:23:11.556428+0000","last_peered":"2024-05-11T12:23:11.556428+0000","last_clean":"2024-05-11T12:23:11.556428+0000","last_became_active":"2024-05-11T12:21:11.529580+0000","last_became_peered":"2024-05-11T12:21:11.529580+0000","last_unstale":"2024-05-11T12:23:11.556428+0000","last_undegraded":"2024-05-11T12:23:11.556428+0000","last_fullsized":"2024-05-11T12:23:11.556428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:29.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094372+0000","last_change":"2024-05-11T12:21:42.674898+0000","last_active":"2024-05-11T12:22:44.094372+0000","last_peered":"2024-05-11T12:22:44.094372+0000","last_clean":"2024-05-11T12:22:44.094372+0000","last_became_active":"2024-05-11T12:21:42.669853+0000","last_became_peered":"2024-05-11T12:21:42.669853+0000","last_unstale":"2024-05-11T12:22:44.094372+0000","last_undegraded":"2024-05-11T12:22:44.094372+0000","last_fullsized":"2024-05-11T12:22:44.094372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:36.040786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559948+0000","last_change":"2024-05-11T12:21:11.534706+0000","last_active":"2024-05-11T12:23:11.559948+0000","last_peered":"2024-05-11T12:23:11.559948+0000","last_clean":"2024-05-11T12:23:11.559948+0000","last_became_active":"2024-05-11T12:21:11.534301+0000","last_became_peered":"2024-05-11T12:21:11.534301+0000","last_unstale":"2024-05-11T12:23:11.559948+0000","last_undegraded":"2024-05-11T12:23:11.559948+0000","last_fullsized":"2024-05-11T12:23:11.559948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:24.277988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094123+0000","last_change":"2024-05-11T12:21:08.513794+0000","last_active":"2024-05-11T12:22:44.094123+0000","last_peered":"2024-05-11T12:22:44.094123+0000","last_clean":"2024-05-11T12:22:44.094123+0000","last_became_active":"2024-05-11T12:21:08.513383+0000","last_became_peered":"2024-05-11T12:21:08.513383+0000","last_unstale":"2024-05-11T12:22:44.094123+0000","last_undegraded":"2024-05-11T12:22:44.094123+0000","last_fullsized":"2024-05-11T12:22:44.094123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:50.558886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562588+0000","last_change":"2024-05-11T12:21:11.537147+0000","last_active":"2024-05-11T12:23:11.562588+0000","last_peered":"2024-05-11T12:23:11.562588+0000","last_clean":"2024-05-11T12:23:11.562588+0000","last_became_active":"2024-05-11T12:21:11.536739+0000","last_became_peered":"2024-05-11T12:21:11.536739+0000","last_unstale":"2024-05-11T12:23:11.562588+0000","last_undegraded":"2024-05-11T12:23:11.562588+0000","last_fullsized":"2024-05-11T12:23:11.562588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:00:45.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751625+0000","last_change":"2024-05-11T12:21:08.513494+0000","last_active":"2024-05-11T12:22:43.751625+0000","last_peered":"2024-05-11T12:22:43.751625+0000","last_clean":"2024-05-11T12:22:43.751625+0000","last_became_active":"2024-05-11T12:21:08.513089+0000","last_became_peered":"2024-05-11T12:21:08.513089+0000","last_unstale":"2024-05-11T12:22:43.751625+0000","last_undegraded":"2024-05-11T12:22:43.751625+0000","last_fullsized":"2024-05-11T12:22:43.751625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:48:28.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.536678+0000","last_change":"2024-05-11T12:21:11.527827+0000","last_active":"2024-05-11T12:23:11.536678+0000","last_peered":"2024-05-11T12:23:11.536678+0000","last_clean":"2024-05-11T12:23:11.536678+0000","last_became_active":"2024-05-11T12:21:11.527397+0000","last_became_peered":"2024-05-11T12:21:11.527397+0000","last_unstale":"2024-05-11T12:23:11.536678+0000","last_undegraded":"2024-05-11T12:23:11.536678+0000","last_fullsized":"2024-05-11T12:23:11.536678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:18.710855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748805+0000","last_change":"2024-05-11T12:21:08.513519+0000","last_active":"2024-05-11T12:22:43.748805+0000","last_peered":"2024-05-11T12:22:43.748805+0000","last_clean":"2024-05-11T12:22:43.748805+0000","last_became_active":"2024-05-11T12:21:08.512566+0000","last_became_peered":"2024-05-11T12:21:08.512566+0000","last_unstale":"2024-05-11T12:22:43.748805+0000","last_undegraded":"2024-05-11T12:22:43.748805+0000","last_fullsized":"2024-05-11T12:22:43.748805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:56.104043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562028+0000","last_change":"2024-05-11T12:21:11.535364+0000","last_active":"2024-05-11T12:23:11.562028+0000","last_peered":"2024-05-11T12:23:11.562028+0000","last_clean":"2024-05-11T12:23:11.562028+0000","last_became_active":"2024-05-11T12:21:11.534988+0000","last_became_peered":"2024-05-11T12:21:11.534988+0000","last_unstale":"2024-05-11T12:23:11.562028+0000","last_undegraded":"2024-05-11T12:23:11.562028+0000","last_fullsized":"2024-05-11T12:23:11.562028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:25:20.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844753+0000","last_change":"2024-05-11T12:21:08.509913+0000","last_active":"2024-05-11T12:22:43.844753+0000","last_peered":"2024-05-11T12:22:43.844753+0000","last_clean":"2024-05-11T12:22:43.844753+0000","last_became_active":"2024-05-11T12:21:08.509498+0000","last_became_peered":"2024-05-11T12:21:08.509498+0000","last_unstale":"2024-05-11T12:22:43.844753+0000","last_undegraded":"2024-05-11T12:22:43.844753+0000","last_fullsized":"2024-05-11T12:22:43.844753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:43.282700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540114+0000","last_change":"2024-05-11T12:21:11.533108+0000","last_active":"2024-05-11T12:23:11.540114+0000","last_peered":"2024-05-11T12:23:11.540114+0000","last_clean":"2024-05-11T12:23:11.540114+0000","last_became_active":"2024-05-11T12:21:11.532779+0000","last_became_peered":"2024-05-11T12:21:11.532779+0000","last_unstale":"2024-05-11T12:23:11.540114+0000","last_undegraded":"2024-05-11T12:23:11.540114+0000","last_fullsized":"2024-05-11T12:23:11.540114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:55.311786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751328+0000","last_change":"2024-05-11T12:21:08.513355+0000","last_active":"2024-05-11T12:22:43.751328+0000","last_peered":"2024-05-11T12:22:43.751328+0000","last_clean":"2024-05-11T12:22:43.751328+0000","last_became_active":"2024-05-11T12:21:08.512964+0000","last_became_peered":"2024-05-11T12:21:08.512964+0000","last_unstale":"2024-05-11T12:22:43.751328+0000","last_undegraded":"2024-05-11T12:22:43.751328+0000","last_fullsized":"2024-05-11T12:22:43.751328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:12.296865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548024+0000","last_change":"2024-05-11T12:21:11.525958+0000","last_active":"2024-05-11T12:23:11.548024+0000","last_peered":"2024-05-11T12:23:11.548024+0000","last_clean":"2024-05-11T12:23:11.548024+0000","last_became_active":"2024-05-11T12:21:11.525285+0000","last_became_peered":"2024-05-11T12:21:11.525285+0000","last_unstale":"2024-05-11T12:23:11.548024+0000","last_undegraded":"2024-05-11T12:23:11.548024+0000","last_fullsized":"2024-05-11T12:23:11.548024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:24:47.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.749227+0000","last_change":"2024-05-11T12:21:42.673378+0000","last_active":"2024-05-11T12:22:43.749227+0000","last_peered":"2024-05-11T12:22:43.749227+0000","last_clean":"2024-05-11T12:22:43.749227+0000","last_became_active":"2024-05-11T12:21:42.672870+0000","last_became_peered":"2024-05-11T12:21:42.672870+0000","last_unstale":"2024-05-11T12:22:43.749227+0000","last_undegraded":"2024-05-11T12:22:43.749227+0000","last_fullsized":"2024-05-11T12:22:43.749227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:55:41.845162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559484+0000","last_change":"2024-05-11T12:21:11.538946+0000","last_active":"2024-05-11T12:23:11.559484+0000","last_peered":"2024-05-11T12:23:11.559484+0000","last_clean":"2024-05-11T12:23:11.559484+0000","last_became_active":"2024-05-11T12:21:11.535804+0000","last_became_peered":"2024-05-11T12:21:11.535804+0000","last_unstale":"2024-05-11T12:23:11.559484+0000","last_undegraded":"2024-05-11T12:23:11.559484+0000","last_fullsized":"2024-05-11T12:23:11.559484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:37:16.221144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649463+0000","last_change":"2024-05-11T12:21:42.674203+0000","last_active":"2024-05-11T12:22:43.649463+0000","last_peered":"2024-05-11T12:22:43.649463+0000","last_clean":"2024-05-11T12:22:43.649463+0000","last_became_active":"2024-05-11T12:21:42.673478+0000","last_became_peered":"2024-05-11T12:21:42.673478+0000","last_unstale":"2024-05-11T12:22:43.649463+0000","last_undegraded":"2024-05-11T12:22:43.649463+0000","last_fullsized":"2024-05-11T12:22:43.649463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:20.648528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553883+0000","last_change":"2024-05-11T12:21:11.535346+0000","last_active":"2024-05-11T12:23:11.553883+0000","last_peered":"2024-05-11T12:23:11.553883+0000","last_clean":"2024-05-11T12:23:11.553883+0000","last_became_active":"2024-05-11T12:21:11.530262+0000","last_became_peered":"2024-05-11T12:21:11.530262+0000","last_unstale":"2024-05-11T12:23:11.553883+0000","last_undegraded":"2024-05-11T12:23:11.553883+0000","last_fullsized":"2024-05-11T12:23:11.553883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:13.169797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.752106+0000","last_change":"2024-05-11T12:21:08.531598+0000","last_active":"2024-05-11T12:22:43.752106+0000","last_peered":"2024-05-11T12:22:43.752106+0000","last_clean":"2024-05-11T12:22:43.752106+0000","last_became_active":"2024-05-11T12:21:08.531158+0000","last_became_peered":"2024-05-11T12:21:08.531158+0000","last_unstale":"2024-05-11T12:22:43.752106+0000","last_undegraded":"2024-05-11T12:22:43.752106+0000","last_fullsized":"2024-05-11T12:22:43.752106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:43.528151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544607+0000","last_change":"2024-05-11T12:22:44.094901+0000","last_active":"2024-05-11T12:23:11.544607+0000","last_peered":"2024-05-11T12:23:11.544607+0000","last_clean":"2024-05-11T12:23:11.544607+0000","last_became_active":"2024-05-11T12:22:44.094563+0000","last_became_peered":"2024-05-11T12:22:44.094563+0000","last_unstale":"2024-05-11T12:23:11.544607+0000","last_undegraded":"2024-05-11T12:23:11.544607+0000","last_fullsized":"2024-05-11T12:23:11.544607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:42:41.499957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,4],"acting":[9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'33","reported_seq":389,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:24:03.871569+0000","last_change":"2024-05-11T12:21:08.516737+0000","last_active":"2024-05-11T12:24:03.871569+0000","last_peered":"2024-05-11T12:24:03.871569+0000","last_clean":"2024-05-11T12:24:03.871569+0000","last_became_active":"2024-05-11T12:21:08.516316+0000","last_became_peered":"2024-05-11T12:21:08.516316+0000","last_unstale":"2024-05-11T12:24:03.871569+0000","last_undegraded":"2024-05-11T12:24:03.871569+0000","last_fullsized":"2024-05-11T12:24:03.871569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:25.769713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62307,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":325,"num_read_kb":223,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545930+0000","last_change":"2024-05-11T12:21:11.528553+0000","last_active":"2024-05-11T12:23:11.545930+0000","last_peered":"2024-05-11T12:23:11.545930+0000","last_clean":"2024-05-11T12:23:11.545930+0000","last_became_active":"2024-05-11T12:21:11.528175+0000","last_became_peered":"2024-05-11T12:21:11.528175+0000","last_unstale":"2024-05-11T12:23:11.545930+0000","last_undegraded":"2024-05-11T12:23:11.545930+0000","last_fullsized":"2024-05-11T12:23:11.545930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:46:42.689851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651107+0000","last_change":"2024-05-11T12:21:42.673360+0000","last_active":"2024-05-11T12:22:43.651107+0000","last_peered":"2024-05-11T12:22:43.651107+0000","last_clean":"2024-05-11T12:22:43.651107+0000","last_became_active":"2024-05-11T12:21:42.664842+0000","last_became_peered":"2024-05-11T12:21:42.664842+0000","last_unstale":"2024-05-11T12:22:43.651107+0000","last_undegraded":"2024-05-11T12:22:43.651107+0000","last_fullsized":"2024-05-11T12:22:43.651107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:40.271658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.552610+0000","last_change":"2024-05-11T12:21:11.533157+0000","last_active":"2024-05-11T12:23:11.552610+0000","last_peered":"2024-05-11T12:23:11.552610+0000","last_clean":"2024-05-11T12:23:11.552610+0000","last_became_active":"2024-05-11T12:21:11.532814+0000","last_became_peered":"2024-05-11T12:21:11.532814+0000","last_unstale":"2024-05-11T12:23:11.552610+0000","last_undegraded":"2024-05-11T12:23:11.552610+0000","last_fullsized":"2024-05-11T12:23:11.552610+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:30:56.555384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751604+0000","last_change":"2024-05-11T12:21:08.511907+0000","last_active":"2024-05-11T12:22:43.751604+0000","last_peered":"2024-05-11T12:22:43.751604+0000","last_clean":"2024-05-11T12:22:43.751604+0000","last_became_active":"2024-05-11T12:21:08.511296+0000","last_became_peered":"2024-05-11T12:21:08.511296+0000","last_unstale":"2024-05-11T12:22:43.751604+0000","last_undegraded":"2024-05-11T12:22:43.751604+0000","last_fullsized":"2024-05-11T12:22:43.751604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:43:17.374216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562158+0000","last_change":"2024-05-11T12:21:11.537859+0000","last_active":"2024-05-11T12:23:11.562158+0000","last_peered":"2024-05-11T12:23:11.562158+0000","last_clean":"2024-05-11T12:23:11.562158+0000","last_became_active":"2024-05-11T12:21:11.537448+0000","last_became_peered":"2024-05-11T12:21:11.537448+0000","last_unstale":"2024-05-11T12:23:11.562158+0000","last_undegraded":"2024-05-11T12:23:11.562158+0000","last_fullsized":"2024-05-11T12:23:11.562158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:09:33.866539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647911+0000","last_change":"2024-05-11T12:21:08.510228+0000","last_active":"2024-05-11T12:22:43.647911+0000","last_peered":"2024-05-11T12:22:43.647911+0000","last_clean":"2024-05-11T12:22:43.647911+0000","last_became_active":"2024-05-11T12:21:08.509671+0000","last_became_peered":"2024-05-11T12:21:08.509671+0000","last_unstale":"2024-05-11T12:22:43.647911+0000","last_undegraded":"2024-05-11T12:22:43.647911+0000","last_fullsized":"2024-05-11T12:22:43.647911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:06.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556927+0000","last_change":"2024-05-11T12:21:11.530935+0000","last_active":"2024-05-11T12:23:11.556927+0000","last_peered":"2024-05-11T12:23:11.556927+0000","last_clean":"2024-05-11T12:23:11.556927+0000","last_became_active":"2024-05-11T12:21:11.530579+0000","last_became_peered":"2024-05-11T12:21:11.530579+0000","last_unstale":"2024-05-11T12:23:11.556927+0000","last_undegraded":"2024-05-11T12:23:11.556927+0000","last_fullsized":"2024-05-11T12:23:11.556927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:38.296021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'8","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651091+0000","last_change":"2024-05-11T12:21:42.681114+0000","last_active":"2024-05-11T12:22:43.651091+0000","last_peered":"2024-05-11T12:22:43.651091+0000","last_clean":"2024-05-11T12:22:43.651091+0000","last_became_active":"2024-05-11T12:21:42.666878+0000","last_became_peered":"2024-05-11T12:21:42.666878+0000","last_unstale":"2024-05-11T12:22:43.651091+0000","last_undegraded":"2024-05-11T12:22:43.651091+0000","last_fullsized":"2024-05-11T12:22:43.651091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:45.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547859+0000","last_change":"2024-05-11T12:22:43.892989+0000","last_active":"2024-05-11T12:23:11.547859+0000","last_peered":"2024-05-11T12:23:11.547859+0000","last_clean":"2024-05-11T12:23:11.547859+0000","last_became_active":"2024-05-11T12:22:43.892599+0000","last_became_peered":"2024-05-11T12:22:43.892599+0000","last_unstale":"2024-05-11T12:23:11.547859+0000","last_undegraded":"2024-05-11T12:23:11.547859+0000","last_fullsized":"2024-05-11T12:23:11.547859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:46.357968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094221+0000","last_change":"2024-05-11T12:21:08.510504+0000","last_active":"2024-05-11T12:22:44.094221+0000","last_peered":"2024-05-11T12:22:44.094221+0000","last_clean":"2024-05-11T12:22:44.094221+0000","last_became_active":"2024-05-11T12:21:08.510074+0000","last_became_peered":"2024-05-11T12:21:08.510074+0000","last_unstale":"2024-05-11T12:22:44.094221+0000","last_undegraded":"2024-05-11T12:22:44.094221+0000","last_fullsized":"2024-05-11T12:22:44.094221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:31:49.465252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.555652+0000","last_change":"2024-05-11T12:22:43.753140+0000","last_active":"2024-05-11T12:23:11.555652+0000","last_peered":"2024-05-11T12:23:11.555652+0000","last_clean":"2024-05-11T12:23:11.555652+0000","last_became_active":"2024-05-11T12:22:43.752693+0000","last_became_peered":"2024-05-11T12:22:43.752693+0000","last_unstale":"2024-05-11T12:23:11.555652+0000","last_undegraded":"2024-05-11T12:23:11.555652+0000","last_fullsized":"2024-05-11T12:23:11.555652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:13.152017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647507+0000","last_change":"2024-05-11T12:21:08.531926+0000","last_active":"2024-05-11T12:22:43.647507+0000","last_peered":"2024-05-11T12:22:43.647507+0000","last_clean":"2024-05-11T12:22:43.647507+0000","last_became_active":"2024-05-11T12:21:08.531548+0000","last_became_peered":"2024-05-11T12:21:08.531548+0000","last_unstale":"2024-05-11T12:22:43.647507+0000","last_undegraded":"2024-05-11T12:22:43.647507+0000","last_fullsized":"2024-05-11T12:22:43.647507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:25:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548090+0000","last_change":"2024-05-11T12:21:11.528595+0000","last_active":"2024-05-11T12:23:11.548090+0000","last_peered":"2024-05-11T12:23:11.548090+0000","last_clean":"2024-05-11T12:23:11.548090+0000","last_became_active":"2024-05-11T12:21:11.527782+0000","last_became_peered":"2024-05-11T12:21:11.527782+0000","last_unstale":"2024-05-11T12:23:11.548090+0000","last_undegraded":"2024-05-11T12:23:11.548090+0000","last_fullsized":"2024-05-11T12:23:11.548090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:31:00.337497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649651+0000","last_change":"2024-05-11T12:21:08.508093+0000","last_active":"2024-05-11T12:22:43.649651+0000","last_peered":"2024-05-11T12:22:43.649651+0000","last_clean":"2024-05-11T12:22:43.649651+0000","last_became_active":"2024-05-11T12:21:08.507710+0000","last_became_peered":"2024-05-11T12:21:08.507710+0000","last_unstale":"2024-05-11T12:22:43.649651+0000","last_undegraded":"2024-05-11T12:22:43.649651+0000","last_fullsized":"2024-05-11T12:22:43.649651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:24.273932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.557566+0000","last_change":"2024-05-11T12:21:11.526026+0000","last_active":"2024-05-11T12:23:11.557566+0000","last_peered":"2024-05-11T12:23:11.557566+0000","last_clean":"2024-05-11T12:23:11.557566+0000","last_became_active":"2024-05-11T12:21:11.525628+0000","last_became_peered":"2024-05-11T12:21:11.525628+0000","last_unstale":"2024-05-11T12:23:11.557566+0000","last_undegraded":"2024-05-11T12:23:11.557566+0000","last_fullsized":"2024-05-11T12:23:11.557566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:12.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561475+0000","last_change":"2024-05-11T12:21:11.535808+0000","last_active":"2024-05-11T12:23:11.561475+0000","last_peered":"2024-05-11T12:23:11.561475+0000","last_clean":"2024-05-11T12:23:11.561475+0000","last_became_active":"2024-05-11T12:21:11.535204+0000","last_became_peered":"2024-05-11T12:21:11.535204+0000","last_unstale":"2024-05-11T12:23:11.561475+0000","last_undegraded":"2024-05-11T12:23:11.561475+0000","last_fullsized":"2024-05-11T12:23:11.561475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:36:01.208543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844644+0000","last_change":"2024-05-11T12:21:08.531838+0000","last_active":"2024-05-11T12:22:43.844644+0000","last_peered":"2024-05-11T12:22:43.844644+0000","last_clean":"2024-05-11T12:22:43.844644+0000","last_became_active":"2024-05-11T12:21:08.531491+0000","last_became_peered":"2024-05-11T12:21:08.531491+0000","last_unstale":"2024-05-11T12:22:43.844644+0000","last_undegraded":"2024-05-11T12:22:43.844644+0000","last_fullsized":"2024-05-11T12:22:43.844644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:33.928985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.535726+0000","last_change":"2024-05-11T12:21:11.538563+0000","last_active":"2024-05-11T12:23:11.535726+0000","last_peered":"2024-05-11T12:23:11.535726+0000","last_clean":"2024-05-11T12:23:11.535726+0000","last_became_active":"2024-05-11T12:21:11.538200+0000","last_became_peered":"2024-05-11T12:21:11.538200+0000","last_unstale":"2024-05-11T12:23:11.535726+0000","last_undegraded":"2024-05-11T12:23:11.535726+0000","last_fullsized":"2024-05-11T12:23:11.535726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:18.879528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648409+0000","last_change":"2024-05-11T12:21:08.509972+0000","last_active":"2024-05-11T12:22:43.648409+0000","last_peered":"2024-05-11T12:22:43.648409+0000","last_clean":"2024-05-11T12:22:43.648409+0000","last_became_active":"2024-05-11T12:21:08.509531+0000","last_became_peered":"2024-05-11T12:21:08.509531+0000","last_unstale":"2024-05-11T12:22:43.648409+0000","last_undegraded":"2024-05-11T12:22:43.648409+0000","last_fullsized":"2024-05-11T12:22:43.648409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:47:13.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649321+0000","last_change":"2024-05-11T12:19:42.546860+0000","last_active":"2024-05-11T12:22:43.649321+0000","last_peered":"2024-05-11T12:22:43.649321+0000","last_clean":"2024-05-11T12:22:43.649321+0000","last_became_active":"2024-05-11T12:19:42.541449+0000","last_became_peered":"2024-05-11T12:19:42.541449+0000","last_unstale":"2024-05-11T12:22:43.649321+0000","last_undegraded":"2024-05-11T12:22:43.649321+0000","last_fullsized":"2024-05-11T12:22:43.649321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_clean_scrub_stamp":"2024-05-11T12:16:12.381187+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:45.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559597+0000","last_change":"2024-05-11T12:21:11.536770+0000","last_active":"2024-05-11T12:23:11.559597+0000","last_peered":"2024-05-11T12:23:11.559597+0000","last_clean":"2024-05-11T12:23:11.559597+0000","last_became_active":"2024-05-11T12:21:11.536421+0000","last_became_peered":"2024-05-11T12:21:11.536421+0000","last_unstale":"2024-05-11T12:23:11.559597+0000","last_undegraded":"2024-05-11T12:23:11.559597+0000","last_fullsized":"2024-05-11T12:23:11.559597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:42.217398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649356+0000","last_change":"2024-05-11T12:21:42.675283+0000","last_active":"2024-05-11T12:22:43.649356+0000","last_peered":"2024-05-11T12:22:43.649356+0000","last_clean":"2024-05-11T12:22:43.649356+0000","last_became_active":"2024-05-11T12:21:42.674775+0000","last_became_peered":"2024-05-11T12:21:42.674775+0000","last_unstale":"2024-05-11T12:22:43.649356+0000","last_undegraded":"2024-05-11T12:22:43.649356+0000","last_fullsized":"2024-05-11T12:22:43.649356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:13:30.143988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094224+0000","last_change":"2024-05-11T12:21:08.533462+0000","last_active":"2024-05-11T12:22:44.094224+0000","last_peered":"2024-05-11T12:22:44.094224+0000","last_clean":"2024-05-11T12:22:44.094224+0000","last_became_active":"2024-05-11T12:21:08.533081+0000","last_became_peered":"2024-05-11T12:21:08.533081+0000","last_unstale":"2024-05-11T12:22:44.094224+0000","last_undegraded":"2024-05-11T12:22:44.094224+0000","last_fullsized":"2024-05-11T12:22:44.094224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:01:48.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.549932+0000","last_change":"2024-05-11T12:21:11.539484+0000","last_active":"2024-05-11T12:23:11.549932+0000","last_peered":"2024-05-11T12:23:11.549932+0000","last_clean":"2024-05-11T12:23:11.549932+0000","last_became_active":"2024-05-11T12:21:11.536810+0000","last_became_peered":"2024-05-11T12:21:11.536810+0000","last_unstale":"2024-05-11T12:23:11.549932+0000","last_undegraded":"2024-05-11T12:23:11.549932+0000","last_fullsized":"2024-05-11T12:23:11.549932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:47:41.461766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.614778+0000","last_change":"2024-05-11T12:21:08.510519+0000","last_active":"2024-05-11T12:23:11.614778+0000","last_peered":"2024-05-11T12:23:11.614778+0000","last_clean":"2024-05-11T12:23:11.614778+0000","last_became_active":"2024-05-11T12:21:08.510180+0000","last_became_peered":"2024-05-11T12:21:08.510180+0000","last_unstale":"2024-05-11T12:23:11.614778+0000","last_undegraded":"2024-05-11T12:23:11.614778+0000","last_fullsized":"2024-05-11T12:23:11.614778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:24.537043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553050+0000","last_change":"2024-05-11T12:21:11.533726+0000","last_active":"2024-05-11T12:23:11.553050+0000","last_peered":"2024-05-11T12:23:11.553050+0000","last_clean":"2024-05-11T12:23:11.553050+0000","last_became_active":"2024-05-11T12:21:11.533393+0000","last_became_peered":"2024-05-11T12:21:11.533393+0000","last_unstale":"2024-05-11T12:23:11.553050+0000","last_undegraded":"2024-05-11T12:23:11.553050+0000","last_fullsized":"2024-05-11T12:23:11.553050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:03.071666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648390+0000","last_change":"2024-05-11T12:21:09.036622+0000","last_active":"2024-05-11T12:22:43.648390+0000","last_peered":"2024-05-11T12:22:43.648390+0000","last_clean":"2024-05-11T12:22:43.648390+0000","last_became_active":"2024-05-11T12:21:09.036209+0000","last_became_peered":"2024-05-11T12:21:09.036209+0000","last_unstale":"2024-05-11T12:22:43.648390+0000","last_undegraded":"2024-05-11T12:22:43.648390+0000","last_fullsized":"2024-05-11T12:22:43.648390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:29:58.947588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553776+0000","last_change":"2024-05-11T12:22:43.846111+0000","last_active":"2024-05-11T12:23:11.553776+0000","last_peered":"2024-05-11T12:23:11.553776+0000","last_clean":"2024-05-11T12:23:11.553776+0000","last_became_active":"2024-05-11T12:22:43.845712+0000","last_became_peered":"2024-05-11T12:22:43.845712+0000","last_unstale":"2024-05-11T12:23:11.553776+0000","last_undegraded":"2024-05-11T12:23:11.553776+0000","last_fullsized":"2024-05-11T12:23:11.553776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:19:47.516917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891608+0000","last_change":"2024-05-11T12:21:08.533435+0000","last_active":"2024-05-11T12:22:43.891608+0000","last_peered":"2024-05-11T12:22:43.891608+0000","last_clean":"2024-05-11T12:22:43.891608+0000","last_became_active":"2024-05-11T12:21:08.533100+0000","last_became_peered":"2024-05-11T12:21:08.533100+0000","last_unstale":"2024-05-11T12:22:43.891608+0000","last_undegraded":"2024-05-11T12:22:43.891608+0000","last_fullsized":"2024-05-11T12:22:43.891608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:17.619015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.529373+0000","last_change":"2024-05-11T12:21:11.526140+0000","last_active":"2024-05-11T12:23:11.529373+0000","last_peered":"2024-05-11T12:23:11.529373+0000","last_clean":"2024-05-11T12:23:11.529373+0000","last_became_active":"2024-05-11T12:21:11.525810+0000","last_became_peered":"2024-05-11T12:21:11.525810+0000","last_unstale":"2024-05-11T12:23:11.529373+0000","last_undegraded":"2024-05-11T12:23:11.529373+0000","last_fullsized":"2024-05-11T12:23:11.529373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:14:04.160379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790116+0000","last_change":"2024-05-11T12:21:08.508031+0000","last_active":"2024-05-11T12:22:43.790116+0000","last_peered":"2024-05-11T12:22:43.790116+0000","last_clean":"2024-05-11T12:22:43.790116+0000","last_became_active":"2024-05-11T12:21:08.507703+0000","last_became_peered":"2024-05-11T12:21:08.507703+0000","last_unstale":"2024-05-11T12:22:43.790116+0000","last_undegraded":"2024-05-11T12:22:43.790116+0000","last_fullsized":"2024-05-11T12:22:43.790116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:05.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547615+0000","last_change":"2024-05-11T12:21:11.528293+0000","last_active":"2024-05-11T12:23:11.547615+0000","last_peered":"2024-05-11T12:23:11.547615+0000","last_clean":"2024-05-11T12:23:11.547615+0000","last_became_active":"2024-05-11T12:21:11.527862+0000","last_became_peered":"2024-05-11T12:21:11.527862+0000","last_unstale":"2024-05-11T12:23:11.547615+0000","last_undegraded":"2024-05-11T12:23:11.547615+0000","last_fullsized":"2024-05-11T12:23:11.547615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:07.335829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653717+0000","last_change":"2024-05-11T12:21:08.510138+0000","last_active":"2024-05-11T12:22:43.653717+0000","last_peered":"2024-05-11T12:22:43.653717+0000","last_clean":"2024-05-11T12:22:43.653717+0000","last_became_active":"2024-05-11T12:21:08.509566+0000","last_became_peered":"2024-05-11T12:21:08.509566+0000","last_unstale":"2024-05-11T12:22:43.653717+0000","last_undegraded":"2024-05-11T12:22:43.653717+0000","last_fullsized":"2024-05-11T12:22:43.653717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:24:58.174305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.542051+0000","last_change":"2024-05-11T12:22:43.750230+0000","last_active":"2024-05-11T12:23:11.542051+0000","last_peered":"2024-05-11T12:23:11.542051+0000","last_clean":"2024-05-11T12:23:11.542051+0000","last_became_active":"2024-05-11T12:22:43.749836+0000","last_became_peered":"2024-05-11T12:22:43.749836+0000","last_unstale":"2024-05-11T12:23:11.542051+0000","last_undegraded":"2024-05-11T12:23:11.542051+0000","last_fullsized":"2024-05-11T12:23:11.542051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:36.498862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"79'2","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093949+0000","last_change":"2024-05-11T12:21:42.683706+0000","last_active":"2024-05-11T12:22:44.093949+0000","last_peered":"2024-05-11T12:22:44.093949+0000","last_clean":"2024-05-11T12:22:44.093949+0000","last_became_active":"2024-05-11T12:21:42.675791+0000","last_became_peered":"2024-05-11T12:21:42.675791+0000","last_unstale":"2024-05-11T12:22:44.093949+0000","last_undegraded":"2024-05-11T12:22:44.093949+0000","last_fullsized":"2024-05-11T12:22:44.093949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:13:26.737525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4],"acting":[8,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560452+0000","last_change":"2024-05-11T12:21:11.544153+0000","last_active":"2024-05-11T12:23:11.560452+0000","last_peered":"2024-05-11T12:23:11.560452+0000","last_clean":"2024-05-11T12:23:11.560452+0000","last_became_active":"2024-05-11T12:21:11.543772+0000","last_became_peered":"2024-05-11T12:21:11.543772+0000","last_unstale":"2024-05-11T12:23:11.560452+0000","last_undegraded":"2024-05-11T12:23:11.560452+0000","last_fullsized":"2024-05-11T12:23:11.560452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:54:30.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653862+0000","last_change":"2024-05-11T12:21:09.036409+0000","last_active":"2024-05-11T12:22:43.653862+0000","last_peered":"2024-05-11T12:22:43.653862+0000","last_clean":"2024-05-11T12:22:43.653862+0000","last_became_active":"2024-05-11T12:21:09.036009+0000","last_became_peered":"2024-05-11T12:21:09.036009+0000","last_unstale":"2024-05-11T12:22:43.653862+0000","last_undegraded":"2024-05-11T12:22:43.653862+0000","last_fullsized":"2024-05-11T12:22:43.653862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:55:28.736718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561529+0000","last_change":"2024-05-11T12:21:11.542685+0000","last_active":"2024-05-11T12:23:11.561529+0000","last_peered":"2024-05-11T12:23:11.561529+0000","last_clean":"2024-05-11T12:23:11.561529+0000","last_became_active":"2024-05-11T12:21:11.542130+0000","last_became_peered":"2024-05-11T12:21:11.542130+0000","last_unstale":"2024-05-11T12:23:11.561529+0000","last_undegraded":"2024-05-11T12:23:11.561529+0000","last_fullsized":"2024-05-11T12:23:11.561529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:22.578809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751961+0000","last_change":"2024-05-11T12:21:08.510869+0000","last_active":"2024-05-11T12:22:43.751961+0000","last_peered":"2024-05-11T12:22:43.751961+0000","last_clean":"2024-05-11T12:22:43.751961+0000","last_became_active":"2024-05-11T12:21:08.510510+0000","last_became_peered":"2024-05-11T12:21:08.510510+0000","last_unstale":"2024-05-11T12:22:43.751961+0000","last_undegraded":"2024-05-11T12:22:43.751961+0000","last_fullsized":"2024-05-11T12:22:43.751961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:50.002313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556787+0000","last_change":"2024-05-11T12:21:11.538622+0000","last_active":"2024-05-11T12:23:11.556787+0000","last_peered":"2024-05-11T12:23:11.556787+0000","last_clean":"2024-05-11T12:23:11.556787+0000","last_became_active":"2024-05-11T12:21:11.538229+0000","last_became_peered":"2024-05-11T12:21:11.538229+0000","last_unstale":"2024-05-11T12:23:11.556787+0000","last_undegraded":"2024-05-11T12:23:11.556787+0000","last_fullsized":"2024-05-11T12:23:11.556787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:01:15.912428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.845101+0000","last_change":"2024-05-11T12:21:42.677421+0000","last_active":"2024-05-11T12:22:43.845101+0000","last_peered":"2024-05-11T12:22:43.845101+0000","last_clean":"2024-05-11T12:22:43.845101+0000","last_became_active":"2024-05-11T12:21:42.672695+0000","last_became_peered":"2024-05-11T12:21:42.672695+0000","last_unstale":"2024-05-11T12:22:43.845101+0000","last_undegraded":"2024-05-11T12:22:43.845101+0000","last_fullsized":"2024-05-11T12:22:43.845101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:30:27.606816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556993+0000","last_change":"2024-05-11T12:21:11.536149+0000","last_active":"2024-05-11T12:23:11.556993+0000","last_peered":"2024-05-11T12:23:11.556993+0000","last_clean":"2024-05-11T12:23:11.556993+0000","last_became_active":"2024-05-11T12:21:11.535763+0000","last_became_peered":"2024-05-11T12:21:11.535763+0000","last_unstale":"2024-05-11T12:23:11.556993+0000","last_undegraded":"2024-05-11T12:23:11.556993+0000","last_fullsized":"2024-05-11T12:23:11.556993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:46:15.246610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547785+0000","last_change":"2024-05-11T12:21:11.534318+0000","last_active":"2024-05-11T12:23:11.547785+0000","last_peered":"2024-05-11T12:23:11.547785+0000","last_clean":"2024-05-11T12:23:11.547785+0000","last_became_active":"2024-05-11T12:21:11.533949+0000","last_became_peered":"2024-05-11T12:21:11.533949+0000","last_unstale":"2024-05-11T12:23:11.547785+0000","last_undegraded":"2024-05-11T12:23:11.547785+0000","last_fullsized":"2024-05-11T12:23:11.547785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:41:42.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561905+0000","last_change":"2024-05-11T12:21:11.529276+0000","last_active":"2024-05-11T12:23:11.561905+0000","last_peered":"2024-05-11T12:23:11.561905+0000","last_clean":"2024-05-11T12:23:11.561905+0000","last_became_active":"2024-05-11T12:21:11.528857+0000","last_became_peered":"2024-05-11T12:21:11.528857+0000","last_unstale":"2024-05-11T12:23:11.561905+0000","last_undegraded":"2024-05-11T12:23:11.561905+0000","last_fullsized":"2024-05-11T12:23:11.561905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:54.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548623+0000","last_change":"2024-05-11T12:21:11.532342+0000","last_active":"2024-05-11T12:23:11.548623+0000","last_peered":"2024-05-11T12:23:11.548623+0000","last_clean":"2024-05-11T12:23:11.548623+0000","last_became_active":"2024-05-11T12:21:11.531949+0000","last_became_peered":"2024-05-11T12:21:11.531949+0000","last_unstale":"2024-05-11T12:23:11.548623+0000","last_undegraded":"2024-05-11T12:23:11.548623+0000","last_fullsized":"2024-05-11T12:23:11.548623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:23:00.010874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548470+0000","last_change":"2024-05-11T12:21:11.531547+0000","last_active":"2024-05-11T12:23:11.548470+0000","last_peered":"2024-05-11T12:23:11.548470+0000","last_clean":"2024-05-11T12:23:11.548470+0000","last_became_active":"2024-05-11T12:21:11.529702+0000","last_became_peered":"2024-05-11T12:21:11.529702+0000","last_unstale":"2024-05-11T12:23:11.548470+0000","last_undegraded":"2024-05-11T12:23:11.548470+0000","last_fullsized":"2024-05-11T12:23:11.548470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:13.857026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.554216+0000","last_change":"2024-05-11T12:21:11.531261+0000","last_active":"2024-05-11T12:23:11.554216+0000","last_peered":"2024-05-11T12:23:11.554216+0000","last_clean":"2024-05-11T12:23:11.554216+0000","last_became_active":"2024-05-11T12:21:11.530561+0000","last_became_peered":"2024-05-11T12:21:11.530561+0000","last_unstale":"2024-05-11T12:23:11.554216+0000","last_undegraded":"2024-05-11T12:23:11.554216+0000","last_fullsized":"2024-05-11T12:23:11.554216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:52:07.654417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.546445+0000","last_change":"2024-05-11T12:21:11.533255+0000","last_active":"2024-05-11T12:23:11.546445+0000","last_peered":"2024-05-11T12:23:11.546445+0000","last_clean":"2024-05-11T12:23:11.546445+0000","last_became_active":"2024-05-11T12:21:11.532909+0000","last_became_peered":"2024-05-11T12:21:11.532909+0000","last_unstale":"2024-05-11T12:23:11.546445+0000","last_undegraded":"2024-05-11T12:23:11.546445+0000","last_fullsized":"2024-05-11T12:23:11.546445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:07:39.501482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553493+0000","last_change":"2024-05-11T12:21:11.528856+0000","last_active":"2024-05-11T12:23:11.553493+0000","last_peered":"2024-05-11T12:23:11.553493+0000","last_clean":"2024-05-11T12:23:11.553493+0000","last_became_active":"2024-05-11T12:21:11.528230+0000","last_became_peered":"2024-05-11T12:21:11.528230+0000","last_unstale":"2024-05-11T12:23:11.553493+0000","last_undegraded":"2024-05-11T12:23:11.553493+0000","last_fullsized":"2024-05-11T12:23:11.553493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:40.429045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553955+0000","last_change":"2024-05-11T12:21:11.534606+0000","last_active":"2024-05-11T12:23:11.553955+0000","last_peered":"2024-05-11T12:23:11.553955+0000","last_clean":"2024-05-11T12:23:11.553955+0000","last_became_active":"2024-05-11T12:21:11.534193+0000","last_became_peered":"2024-05-11T12:21:11.534193+0000","last_unstale":"2024-05-11T12:23:11.553955+0000","last_undegraded":"2024-05-11T12:23:11.553955+0000","last_fullsized":"2024-05-11T12:23:11.553955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:00:12.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548067+0000","last_change":"2024-05-11T12:21:11.525959+0000","last_active":"2024-05-11T12:23:11.548067+0000","last_peered":"2024-05-11T12:23:11.548067+0000","last_clean":"2024-05-11T12:23:11.548067+0000","last_became_active":"2024-05-11T12:21:11.525129+0000","last_became_peered":"2024-05-11T12:21:11.525129+0000","last_unstale":"2024-05-11T12:23:11.548067+0000","last_undegraded":"2024-05-11T12:23:11.548067+0000","last_fullsized":"2024-05-11T12:23:11.548067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:28.828336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547068+0000","last_change":"2024-05-11T12:21:11.538591+0000","last_active":"2024-05-11T12:23:11.547068+0000","last_peered":"2024-05-11T12:23:11.547068+0000","last_clean":"2024-05-11T12:23:11.547068+0000","last_became_active":"2024-05-11T12:21:11.537855+0000","last_became_peered":"2024-05-11T12:21:11.537855+0000","last_unstale":"2024-05-11T12:23:11.547068+0000","last_undegraded":"2024-05-11T12:23:11.547068+0000","last_fullsized":"2024-05-11T12:23:11.547068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:52.625763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.563270+0000","last_change":"2024-05-11T12:21:11.541185+0000","last_active":"2024-05-11T12:23:11.563270+0000","last_peered":"2024-05-11T12:23:11.563270+0000","last_clean":"2024-05-11T12:23:11.563270+0000","last_became_active":"2024-05-11T12:21:11.540799+0000","last_became_peered":"2024-05-11T12:21:11.540799+0000","last_unstale":"2024-05-11T12:23:11.563270+0000","last_undegraded":"2024-05-11T12:23:11.563270+0000","last_fullsized":"2024-05-11T12:23:11.563270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:24.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.552798+0000","last_change":"2024-05-11T12:21:11.527811+0000","last_active":"2024-05-11T12:23:11.552798+0000","last_peered":"2024-05-11T12:23:11.552798+0000","last_clean":"2024-05-11T12:23:11.552798+0000","last_became_active":"2024-05-11T12:21:11.526603+0000","last_became_peered":"2024-05-11T12:21:11.526603+0000","last_unstale":"2024-05-11T12:23:11.552798+0000","last_undegraded":"2024-05-11T12:23:11.552798+0000","last_fullsized":"2024-05-11T12:23:11.552798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:43.833506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561468+0000","last_change":"2024-05-11T12:21:11.530824+0000","last_active":"2024-05-11T12:23:11.561468+0000","last_peered":"2024-05-11T12:23:11.561468+0000","last_clean":"2024-05-11T12:23:11.561468+0000","last_became_active":"2024-05-11T12:21:11.530486+0000","last_became_peered":"2024-05-11T12:21:11.530486+0000","last_unstale":"2024-05-11T12:23:11.561468+0000","last_undegraded":"2024-05-11T12:23:11.561468+0000","last_fullsized":"2024-05-11T12:23:11.561468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:47.054822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247249,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63105,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":331,"num_read_kb":229,"num_write":92,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":189315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":445299,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":1098251,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037822,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445847,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":445869,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430281,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":1036332,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445382,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507572,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":445355,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":507572,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":445603,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445287,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1035932,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:24:09.196 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 -- 172.21.15.47:0/3321228502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98105e60 msgr2=0x7f4b98108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/3321228502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98105e60 0x7f4b98108250 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009900 tx=0x7f4b8c02a410 comp rx=0 tx=0).stop 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 -- 172.21.15.47:0/3321228502 shutdown_connections 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/3321228502 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b98108790 0x7f4b9810abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/3321228502 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98105e60 0x7f4b98108250 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/3321228502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b98103530 0x7f4b98105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 -- 172.21.15.47:0/3321228502 >> 172.21.15.47:0/3321228502 conn(0x7f4b980fcf20 msgr2=0x7f4b980ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 -- 172.21.15.47:0/3321228502 shutdown_connections 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.336+0000 7f4b9cda4640 1 -- 172.21.15.47:0/3321228502 wait complete. 2024-05-11T12:24:09.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 Processor -- start 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 -- start start 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b98103530 0x7f4b98198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b98105e60 0x7f4b98199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98108790 0x7f4b981a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810e5e0 con 0x7f4b98108790 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810e460 con 0x7f4b98103530 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9cda4640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810e760 con 0x7f4b98105e60 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98108790 0x7f4b981a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98108790 0x7f4b981a02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52408/0 (socket says 172.21.15.47:52408) 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 -- 172.21.15.47:0/2250134020 learned_addr learned my addr 172.21.15.47:0/2250134020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b9659c640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b98103530 0x7f4b98198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b95d9b640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b98105e60 0x7f4b98199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 -- 172.21.15.47:0/2250134020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b98105e60 msgr2=0x7f4b98199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b98105e60 0x7f4b98199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 -- 172.21.15.47:0/2250134020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b98103530 msgr2=0x7f4b98198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b98103530 0x7f4b98198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b96d9d640 1 -- 172.21.15.47:0/2250134020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b981a0940 con 0x7f4b98108790 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.337+0000 7f4b95d9b640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b98105e60 0x7f4b98199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.338+0000 7f4b96d9d640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98108790 0x7f4b981a02a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000cad0 tx=0x7f4b9000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.338+0000 7f4b877fe640 1 -- 172.21.15.47:0/2250134020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000ae20 con 0x7f4b98108790 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.338+0000 7f4b877fe640 1 -- 172.21.15.47:0/2250134020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b90005020 con 0x7f4b98108790 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.338+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b981a0c30 con 0x7f4b98108790 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.338+0000 7f4b877fe640 1 -- 172.21.15.47:0/2250134020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000f6b0 con 0x7f4b98108790 2024-05-11T12:24:09.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.338+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b981a1180 con 0x7f4b98108790 2024-05-11T12:24:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.339+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b980fea10 con 0x7f4b98108790 2024-05-11T12:24:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.340+0000 7f4b877fe640 1 -- 172.21.15.47:0/2250134020 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4b900051d0 con 0x7f4b98108790 2024-05-11T12:24:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.341+0000 7f4b877fe640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4b800b27f0 0x7f4b800b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.341+0000 7f4b9659c640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4b800b27f0 0x7f4b800b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.341+0000 7f4b877fe640 1 -- 172.21.15.47:0/2250134020 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4b900e23c0 con 0x7f4b98108790 2024-05-11T12:24:09.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.342+0000 7f4b9659c640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4b800b27f0 0x7f4b800b4cb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4b880058c0 tx=0x7f4b8800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:09.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.346+0000 7f4b877fe640 1 -- 172.21.15.47:0/2250134020 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4b90014030 con 0x7f4b98108790 2024-05-11T12:24:09.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:08 smithi064 ceph-mon[34537]: pgmap v386: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-11T12:24:09.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:08 smithi050 ceph-mon[34072]: pgmap v386: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-05-11T12:24:09.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.534+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b98062ea0 con 0x7f4b800b27f0 2024-05-11T12:24:09.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.541+0000 7f4b877fe640 1 -- 172.21.15.47:0/2250134020 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258997 (secure 0 0 0) 0x7f4b98062ea0 con 0x7f4b800b27f0 2024-05-11T12:24:09.547 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:09.549 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-05-11T12:24:09.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4b800b27f0 msgr2=0x7f4b800b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4b800b27f0 0x7f4b800b4cb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4b880058c0 tx=0x7f4b8800a2d0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98108790 msgr2=0x7f4b981a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98108790 0x7f4b981a02a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000cad0 tx=0x7f4b9000cf90 comp rx=0 tx=0).stop 2024-05-11T12:24:09.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 shutdown_connections 2024-05-11T12:24:09.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4b800b27f0 0x7f4b800b4cb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98108790 0x7f4b981a02a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4b98105e60 0x7f4b98199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 --2- 172.21.15.47:0/2250134020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4b98103530 0x7f4b98198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 >> 172.21.15.47:0/2250134020 conn(0x7f4b980fcf20 msgr2=0x7f4b98103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:09.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 shutdown_connections 2024-05-11T12:24:09.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.544+0000 7f4b9cda4640 1 -- 172.21.15.47:0/2250134020 wait complete. 2024-05-11T12:24:09.556 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-05-11T12:24:07.575917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653707,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":407,"num_read_kb":293,"num_write":217,"num_write_kb":2244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951874,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":670,"ondisk_log_size":670,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393340,"kb_used_data":10908,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124548484,"statfs":{"total":1151940427776,"available":1151537647616,"internally_reserved":0,"allocated":11169792,"data_stored":7304301,"data_compressed":246459,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002973"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553592+0000","last_change":"2024-05-11T12:21:11.536657+0000","last_active":"2024-05-11T12:23:11.553592+0000","last_peered":"2024-05-11T12:23:11.553592+0000","last_clean":"2024-05-11T12:23:11.553592+0000","last_became_active":"2024-05-11T12:21:11.536243+0000","last_became_peered":"2024-05-11T12:21:11.536243+0000","last_unstale":"2024-05-11T12:23:11.553592+0000","last_undegraded":"2024-05-11T12:23:11.553592+0000","last_fullsized":"2024-05-11T12:23:11.553592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:29:20.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545144+0000","last_change":"2024-05-11T12:21:11.527602+0000","last_active":"2024-05-11T12:23:11.545144+0000","last_peered":"2024-05-11T12:23:11.545144+0000","last_clean":"2024-05-11T12:23:11.545144+0000","last_became_active":"2024-05-11T12:21:11.527309+0000","last_became_peered":"2024-05-11T12:21:11.527309+0000","last_unstale":"2024-05-11T12:23:11.545144+0000","last_undegraded":"2024-05-11T12:23:11.545144+0000","last_fullsized":"2024-05-11T12:23:11.545144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:03.764570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556325+0000","last_change":"2024-05-11T12:21:11.529094+0000","last_active":"2024-05-11T12:23:11.556325+0000","last_peered":"2024-05-11T12:23:11.556325+0000","last_clean":"2024-05-11T12:23:11.556325+0000","last_became_active":"2024-05-11T12:21:11.528796+0000","last_became_peered":"2024-05-11T12:21:11.528796+0000","last_unstale":"2024-05-11T12:23:11.556325+0000","last_undegraded":"2024-05-11T12:23:11.556325+0000","last_fullsized":"2024-05-11T12:23:11.556325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:46:41.098066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561517+0000","last_change":"2024-05-11T12:21:11.528903+0000","last_active":"2024-05-11T12:23:11.561517+0000","last_peered":"2024-05-11T12:23:11.561517+0000","last_clean":"2024-05-11T12:23:11.561517+0000","last_became_active":"2024-05-11T12:21:11.528545+0000","last_became_peered":"2024-05-11T12:21:11.528545+0000","last_unstale":"2024-05-11T12:23:11.561517+0000","last_undegraded":"2024-05-11T12:23:11.561517+0000","last_fullsized":"2024-05-11T12:23:11.561517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:56.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544651+0000","last_change":"2024-05-11T12:21:11.521074+0000","last_active":"2024-05-11T12:23:11.544651+0000","last_peered":"2024-05-11T12:23:11.544651+0000","last_clean":"2024-05-11T12:23:11.544651+0000","last_became_active":"2024-05-11T12:21:11.520675+0000","last_became_peered":"2024-05-11T12:21:11.520675+0000","last_unstale":"2024-05-11T12:23:11.544651+0000","last_undegraded":"2024-05-11T12:23:11.544651+0000","last_fullsized":"2024-05-11T12:23:11.544651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:22:50.952687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547969+0000","last_change":"2024-05-11T12:21:42.671243+0000","last_active":"2024-05-11T12:23:11.547969+0000","last_peered":"2024-05-11T12:23:11.547969+0000","last_clean":"2024-05-11T12:23:11.547969+0000","last_became_active":"2024-05-11T12:21:42.670801+0000","last_became_peered":"2024-05-11T12:21:42.670801+0000","last_unstale":"2024-05-11T12:23:11.547969+0000","last_undegraded":"2024-05-11T12:23:11.547969+0000","last_fullsized":"2024-05-11T12:23:11.547969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:02.539855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540731+0000","last_change":"2024-05-11T12:22:44.095706+0000","last_active":"2024-05-11T12:23:11.540731+0000","last_peered":"2024-05-11T12:23:11.540731+0000","last_clean":"2024-05-11T12:23:11.540731+0000","last_became_active":"2024-05-11T12:22:44.095177+0000","last_became_peered":"2024-05-11T12:22:44.095177+0000","last_unstale":"2024-05-11T12:23:11.540731+0000","last_undegraded":"2024-05-11T12:23:11.540731+0000","last_fullsized":"2024-05-11T12:23:11.540731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:11.426607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545528+0000","last_change":"2024-05-11T12:21:11.532874+0000","last_active":"2024-05-11T12:23:11.545528+0000","last_peered":"2024-05-11T12:23:11.545528+0000","last_clean":"2024-05-11T12:23:11.545528+0000","last_became_active":"2024-05-11T12:21:11.532237+0000","last_became_peered":"2024-05-11T12:21:11.532237+0000","last_unstale":"2024-05-11T12:23:11.545528+0000","last_undegraded":"2024-05-11T12:23:11.545528+0000","last_fullsized":"2024-05-11T12:23:11.545528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:13.770719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.557701+0000","last_change":"2024-05-11T12:21:11.536465+0000","last_active":"2024-05-11T12:23:11.557701+0000","last_peered":"2024-05-11T12:23:11.557701+0000","last_clean":"2024-05-11T12:23:11.557701+0000","last_became_active":"2024-05-11T12:21:11.536057+0000","last_became_peered":"2024-05-11T12:21:11.536057+0000","last_unstale":"2024-05-11T12:23:11.557701+0000","last_undegraded":"2024-05-11T12:23:11.557701+0000","last_fullsized":"2024-05-11T12:23:11.557701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:22:45.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560592+0000","last_change":"2024-05-11T12:21:11.528187+0000","last_active":"2024-05-11T12:23:11.560592+0000","last_peered":"2024-05-11T12:23:11.560592+0000","last_clean":"2024-05-11T12:23:11.560592+0000","last_became_active":"2024-05-11T12:21:11.527780+0000","last_became_peered":"2024-05-11T12:21:11.527780+0000","last_unstale":"2024-05-11T12:23:11.560592+0000","last_undegraded":"2024-05-11T12:23:11.560592+0000","last_fullsized":"2024-05-11T12:23:11.560592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:23.484372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560911+0000","last_change":"2024-05-11T12:21:11.543388+0000","last_active":"2024-05-11T12:23:11.560911+0000","last_peered":"2024-05-11T12:23:11.560911+0000","last_clean":"2024-05-11T12:23:11.560911+0000","last_became_active":"2024-05-11T12:21:11.542984+0000","last_became_peered":"2024-05-11T12:21:11.542984+0000","last_unstale":"2024-05-11T12:23:11.560911+0000","last_undegraded":"2024-05-11T12:23:11.560911+0000","last_fullsized":"2024-05-11T12:23:11.560911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:52.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561371+0000","last_change":"2024-05-11T12:21:11.523355+0000","last_active":"2024-05-11T12:23:11.561371+0000","last_peered":"2024-05-11T12:23:11.561371+0000","last_clean":"2024-05-11T12:23:11.561371+0000","last_became_active":"2024-05-11T12:21:11.523007+0000","last_became_peered":"2024-05-11T12:21:11.523007+0000","last_unstale":"2024-05-11T12:23:11.561371+0000","last_undegraded":"2024-05-11T12:23:11.561371+0000","last_fullsized":"2024-05-11T12:23:11.561371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:57.197028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548605+0000","last_change":"2024-05-11T12:21:11.528072+0000","last_active":"2024-05-11T12:23:11.548605+0000","last_peered":"2024-05-11T12:23:11.548605+0000","last_clean":"2024-05-11T12:23:11.548605+0000","last_became_active":"2024-05-11T12:21:11.527478+0000","last_became_peered":"2024-05-11T12:21:11.527478+0000","last_unstale":"2024-05-11T12:23:11.548605+0000","last_undegraded":"2024-05-11T12:23:11.548605+0000","last_fullsized":"2024-05-11T12:23:11.548605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:57:29.001602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544905+0000","last_change":"2024-05-11T12:21:11.526862+0000","last_active":"2024-05-11T12:23:11.544905+0000","last_peered":"2024-05-11T12:23:11.544905+0000","last_clean":"2024-05-11T12:23:11.544905+0000","last_became_active":"2024-05-11T12:21:11.526553+0000","last_became_peered":"2024-05-11T12:21:11.526553+0000","last_unstale":"2024-05-11T12:23:11.544905+0000","last_undegraded":"2024-05-11T12:23:11.544905+0000","last_fullsized":"2024-05-11T12:23:11.544905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:48.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540462+0000","last_change":"2024-05-11T12:21:42.669393+0000","last_active":"2024-05-11T12:23:11.540462+0000","last_peered":"2024-05-11T12:23:11.540462+0000","last_clean":"2024-05-11T12:23:11.540462+0000","last_became_active":"2024-05-11T12:21:42.669000+0000","last_became_peered":"2024-05-11T12:21:42.669000+0000","last_unstale":"2024-05-11T12:23:11.540462+0000","last_undegraded":"2024-05-11T12:23:11.540462+0000","last_fullsized":"2024-05-11T12:23:11.540462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:29.439588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.563015+0000","last_change":"2024-05-11T12:21:11.540698+0000","last_active":"2024-05-11T12:23:11.563015+0000","last_peered":"2024-05-11T12:23:11.563015+0000","last_clean":"2024-05-11T12:23:11.563015+0000","last_became_active":"2024-05-11T12:21:11.539899+0000","last_became_peered":"2024-05-11T12:21:11.539899+0000","last_unstale":"2024-05-11T12:23:11.563015+0000","last_undegraded":"2024-05-11T12:23:11.563015+0000","last_fullsized":"2024-05-11T12:23:11.563015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:48.222664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553880+0000","last_change":"2024-05-11T12:21:11.537055+0000","last_active":"2024-05-11T12:23:11.553880+0000","last_peered":"2024-05-11T12:23:11.553880+0000","last_clean":"2024-05-11T12:23:11.553880+0000","last_became_active":"2024-05-11T12:21:11.536625+0000","last_became_peered":"2024-05-11T12:21:11.536625+0000","last_unstale":"2024-05-11T12:23:11.553880+0000","last_undegraded":"2024-05-11T12:23:11.553880+0000","last_fullsized":"2024-05-11T12:23:11.553880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:33.453467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548280+0000","last_change":"2024-05-11T12:21:11.530254+0000","last_active":"2024-05-11T12:23:11.548280+0000","last_peered":"2024-05-11T12:23:11.548280+0000","last_clean":"2024-05-11T12:23:11.548280+0000","last_became_active":"2024-05-11T12:21:11.529890+0000","last_became_peered":"2024-05-11T12:21:11.529890+0000","last_unstale":"2024-05-11T12:23:11.548280+0000","last_undegraded":"2024-05-11T12:23:11.548280+0000","last_fullsized":"2024-05-11T12:23:11.548280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:36.100931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.549156+0000","last_change":"2024-05-11T12:21:11.532893+0000","last_active":"2024-05-11T12:23:11.549156+0000","last_peered":"2024-05-11T12:23:11.549156+0000","last_clean":"2024-05-11T12:23:11.549156+0000","last_became_active":"2024-05-11T12:21:11.532539+0000","last_became_peered":"2024-05-11T12:21:11.532539+0000","last_unstale":"2024-05-11T12:23:11.549156+0000","last_undegraded":"2024-05-11T12:23:11.549156+0000","last_fullsized":"2024-05-11T12:23:11.549156+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:14:57.888102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'4","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647560+0000","last_change":"2024-05-11T12:21:08.510909+0000","last_active":"2024-05-11T12:22:43.647560+0000","last_peered":"2024-05-11T12:22:43.647560+0000","last_clean":"2024-05-11T12:22:43.647560+0000","last_became_active":"2024-05-11T12:21:08.509986+0000","last_became_peered":"2024-05-11T12:21:08.509986+0000","last_unstale":"2024-05-11T12:22:43.647560+0000","last_undegraded":"2024-05-11T12:22:43.647560+0000","last_fullsized":"2024-05-11T12:22:43.647560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:03:36.430087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547054+0000","last_change":"2024-05-11T12:21:11.532403+0000","last_active":"2024-05-11T12:23:11.547054+0000","last_peered":"2024-05-11T12:23:11.547054+0000","last_clean":"2024-05-11T12:23:11.547054+0000","last_became_active":"2024-05-11T12:21:11.531907+0000","last_became_peered":"2024-05-11T12:21:11.531907+0000","last_unstale":"2024-05-11T12:23:11.547054+0000","last_undegraded":"2024-05-11T12:23:11.547054+0000","last_fullsized":"2024-05-11T12:23:11.547054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:53:27.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790479+0000","last_change":"2024-05-11T12:21:08.531664+0000","last_active":"2024-05-11T12:22:43.790479+0000","last_peered":"2024-05-11T12:22:43.790479+0000","last_clean":"2024-05-11T12:22:43.790479+0000","last_became_active":"2024-05-11T12:21:08.531319+0000","last_became_peered":"2024-05-11T12:21:08.531319+0000","last_unstale":"2024-05-11T12:22:43.790479+0000","last_undegraded":"2024-05-11T12:22:43.790479+0000","last_fullsized":"2024-05-11T12:22:43.790479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:16:08.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556428+0000","last_change":"2024-05-11T12:21:11.532132+0000","last_active":"2024-05-11T12:23:11.556428+0000","last_peered":"2024-05-11T12:23:11.556428+0000","last_clean":"2024-05-11T12:23:11.556428+0000","last_became_active":"2024-05-11T12:21:11.529580+0000","last_became_peered":"2024-05-11T12:21:11.529580+0000","last_unstale":"2024-05-11T12:23:11.556428+0000","last_undegraded":"2024-05-11T12:23:11.556428+0000","last_fullsized":"2024-05-11T12:23:11.556428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:29.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094372+0000","last_change":"2024-05-11T12:21:42.674898+0000","last_active":"2024-05-11T12:22:44.094372+0000","last_peered":"2024-05-11T12:22:44.094372+0000","last_clean":"2024-05-11T12:22:44.094372+0000","last_became_active":"2024-05-11T12:21:42.669853+0000","last_became_peered":"2024-05-11T12:21:42.669853+0000","last_unstale":"2024-05-11T12:22:44.094372+0000","last_undegraded":"2024-05-11T12:22:44.094372+0000","last_fullsized":"2024-05-11T12:22:44.094372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:36.040786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559948+0000","last_change":"2024-05-11T12:21:11.534706+0000","last_active":"2024-05-11T12:23:11.559948+0000","last_peered":"2024-05-11T12:23:11.559948+0000","last_clean":"2024-05-11T12:23:11.559948+0000","last_became_active":"2024-05-11T12:21:11.534301+0000","last_became_peered":"2024-05-11T12:21:11.534301+0000","last_unstale":"2024-05-11T12:23:11.559948+0000","last_undegraded":"2024-05-11T12:23:11.559948+0000","last_fullsized":"2024-05-11T12:23:11.559948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:24.277988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094123+0000","last_change":"2024-05-11T12:21:08.513794+0000","last_active":"2024-05-11T12:22:44.094123+0000","last_peered":"2024-05-11T12:22:44.094123+0000","last_clean":"2024-05-11T12:22:44.094123+0000","last_became_active":"2024-05-11T12:21:08.513383+0000","last_became_peered":"2024-05-11T12:21:08.513383+0000","last_unstale":"2024-05-11T12:22:44.094123+0000","last_undegraded":"2024-05-11T12:22:44.094123+0000","last_fullsized":"2024-05-11T12:22:44.094123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:50.558886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562588+0000","last_change":"2024-05-11T12:21:11.537147+0000","last_active":"2024-05-11T12:23:11.562588+0000","last_peered":"2024-05-11T12:23:11.562588+0000","last_clean":"2024-05-11T12:23:11.562588+0000","last_became_active":"2024-05-11T12:21:11.536739+0000","last_became_peered":"2024-05-11T12:21:11.536739+0000","last_unstale":"2024-05-11T12:23:11.562588+0000","last_undegraded":"2024-05-11T12:23:11.562588+0000","last_fullsized":"2024-05-11T12:23:11.562588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:00:45.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751625+0000","last_change":"2024-05-11T12:21:08.513494+0000","last_active":"2024-05-11T12:22:43.751625+0000","last_peered":"2024-05-11T12:22:43.751625+0000","last_clean":"2024-05-11T12:22:43.751625+0000","last_became_active":"2024-05-11T12:21:08.513089+0000","last_became_peered":"2024-05-11T12:21:08.513089+0000","last_unstale":"2024-05-11T12:22:43.751625+0000","last_undegraded":"2024-05-11T12:22:43.751625+0000","last_fullsized":"2024-05-11T12:22:43.751625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:48:28.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.536678+0000","last_change":"2024-05-11T12:21:11.527827+0000","last_active":"2024-05-11T12:23:11.536678+0000","last_peered":"2024-05-11T12:23:11.536678+0000","last_clean":"2024-05-11T12:23:11.536678+0000","last_became_active":"2024-05-11T12:21:11.527397+0000","last_became_peered":"2024-05-11T12:21:11.527397+0000","last_unstale":"2024-05-11T12:23:11.536678+0000","last_undegraded":"2024-05-11T12:23:11.536678+0000","last_fullsized":"2024-05-11T12:23:11.536678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:18.710855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748805+0000","last_change":"2024-05-11T12:21:08.513519+0000","last_active":"2024-05-11T12:22:43.748805+0000","last_peered":"2024-05-11T12:22:43.748805+0000","last_clean":"2024-05-11T12:22:43.748805+0000","last_became_active":"2024-05-11T12:21:08.512566+0000","last_became_peered":"2024-05-11T12:21:08.512566+0000","last_unstale":"2024-05-11T12:22:43.748805+0000","last_undegraded":"2024-05-11T12:22:43.748805+0000","last_fullsized":"2024-05-11T12:22:43.748805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:56.104043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562028+0000","last_change":"2024-05-11T12:21:11.535364+0000","last_active":"2024-05-11T12:23:11.562028+0000","last_peered":"2024-05-11T12:23:11.562028+0000","last_clean":"2024-05-11T12:23:11.562028+0000","last_became_active":"2024-05-11T12:21:11.534988+0000","last_became_peered":"2024-05-11T12:21:11.534988+0000","last_unstale":"2024-05-11T12:23:11.562028+0000","last_undegraded":"2024-05-11T12:23:11.562028+0000","last_fullsized":"2024-05-11T12:23:11.562028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:25:20.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844753+0000","last_change":"2024-05-11T12:21:08.509913+0000","last_active":"2024-05-11T12:22:43.844753+0000","last_peered":"2024-05-11T12:22:43.844753+0000","last_clean":"2024-05-11T12:22:43.844753+0000","last_became_active":"2024-05-11T12:21:08.509498+0000","last_became_peered":"2024-05-11T12:21:08.509498+0000","last_unstale":"2024-05-11T12:22:43.844753+0000","last_undegraded":"2024-05-11T12:22:43.844753+0000","last_fullsized":"2024-05-11T12:22:43.844753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:43.282700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540114+0000","last_change":"2024-05-11T12:21:11.533108+0000","last_active":"2024-05-11T12:23:11.540114+0000","last_peered":"2024-05-11T12:23:11.540114+0000","last_clean":"2024-05-11T12:23:11.540114+0000","last_became_active":"2024-05-11T12:21:11.532779+0000","last_became_peered":"2024-05-11T12:21:11.532779+0000","last_unstale":"2024-05-11T12:23:11.540114+0000","last_undegraded":"2024-05-11T12:23:11.540114+0000","last_fullsized":"2024-05-11T12:23:11.540114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:55.311786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751328+0000","last_change":"2024-05-11T12:21:08.513355+0000","last_active":"2024-05-11T12:22:43.751328+0000","last_peered":"2024-05-11T12:22:43.751328+0000","last_clean":"2024-05-11T12:22:43.751328+0000","last_became_active":"2024-05-11T12:21:08.512964+0000","last_became_peered":"2024-05-11T12:21:08.512964+0000","last_unstale":"2024-05-11T12:22:43.751328+0000","last_undegraded":"2024-05-11T12:22:43.751328+0000","last_fullsized":"2024-05-11T12:22:43.751328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:12.296865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548024+0000","last_change":"2024-05-11T12:21:11.525958+0000","last_active":"2024-05-11T12:23:11.548024+0000","last_peered":"2024-05-11T12:23:11.548024+0000","last_clean":"2024-05-11T12:23:11.548024+0000","last_became_active":"2024-05-11T12:21:11.525285+0000","last_became_peered":"2024-05-11T12:21:11.525285+0000","last_unstale":"2024-05-11T12:23:11.548024+0000","last_undegraded":"2024-05-11T12:23:11.548024+0000","last_fullsized":"2024-05-11T12:23:11.548024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:24:47.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.749227+0000","last_change":"2024-05-11T12:21:42.673378+0000","last_active":"2024-05-11T12:22:43.749227+0000","last_peered":"2024-05-11T12:22:43.749227+0000","last_clean":"2024-05-11T12:22:43.749227+0000","last_became_active":"2024-05-11T12:21:42.672870+0000","last_became_peered":"2024-05-11T12:21:42.672870+0000","last_unstale":"2024-05-11T12:22:43.749227+0000","last_undegraded":"2024-05-11T12:22:43.749227+0000","last_fullsized":"2024-05-11T12:22:43.749227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:55:41.845162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559484+0000","last_change":"2024-05-11T12:21:11.538946+0000","last_active":"2024-05-11T12:23:11.559484+0000","last_peered":"2024-05-11T12:23:11.559484+0000","last_clean":"2024-05-11T12:23:11.559484+0000","last_became_active":"2024-05-11T12:21:11.535804+0000","last_became_peered":"2024-05-11T12:21:11.535804+0000","last_unstale":"2024-05-11T12:23:11.559484+0000","last_undegraded":"2024-05-11T12:23:11.559484+0000","last_fullsized":"2024-05-11T12:23:11.559484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:37:16.221144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649463+0000","last_change":"2024-05-11T12:21:42.674203+0000","last_active":"2024-05-11T12:22:43.649463+0000","last_peered":"2024-05-11T12:22:43.649463+0000","last_clean":"2024-05-11T12:22:43.649463+0000","last_became_active":"2024-05-11T12:21:42.673478+0000","last_became_peered":"2024-05-11T12:21:42.673478+0000","last_unstale":"2024-05-11T12:22:43.649463+0000","last_undegraded":"2024-05-11T12:22:43.649463+0000","last_fullsized":"2024-05-11T12:22:43.649463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:20.648528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553883+0000","last_change":"2024-05-11T12:21:11.535346+0000","last_active":"2024-05-11T12:23:11.553883+0000","last_peered":"2024-05-11T12:23:11.553883+0000","last_clean":"2024-05-11T12:23:11.553883+0000","last_became_active":"2024-05-11T12:21:11.530262+0000","last_became_peered":"2024-05-11T12:21:11.530262+0000","last_unstale":"2024-05-11T12:23:11.553883+0000","last_undegraded":"2024-05-11T12:23:11.553883+0000","last_fullsized":"2024-05-11T12:23:11.553883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:13.169797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.752106+0000","last_change":"2024-05-11T12:21:08.531598+0000","last_active":"2024-05-11T12:22:43.752106+0000","last_peered":"2024-05-11T12:22:43.752106+0000","last_clean":"2024-05-11T12:22:43.752106+0000","last_became_active":"2024-05-11T12:21:08.531158+0000","last_became_peered":"2024-05-11T12:21:08.531158+0000","last_unstale":"2024-05-11T12:22:43.752106+0000","last_undegraded":"2024-05-11T12:22:43.752106+0000","last_fullsized":"2024-05-11T12:22:43.752106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:43.528151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544607+0000","last_change":"2024-05-11T12:22:44.094901+0000","last_active":"2024-05-11T12:23:11.544607+0000","last_peered":"2024-05-11T12:23:11.544607+0000","last_clean":"2024-05-11T12:23:11.544607+0000","last_became_active":"2024-05-11T12:22:44.094563+0000","last_became_peered":"2024-05-11T12:22:44.094563+0000","last_unstale":"2024-05-11T12:23:11.544607+0000","last_undegraded":"2024-05-11T12:23:11.544607+0000","last_fullsized":"2024-05-11T12:23:11.544607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:42:41.499957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,4],"acting":[9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'33","reported_seq":389,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:24:03.871569+0000","last_change":"2024-05-11T12:21:08.516737+0000","last_active":"2024-05-11T12:24:03.871569+0000","last_peered":"2024-05-11T12:24:03.871569+0000","last_clean":"2024-05-11T12:24:03.871569+0000","last_became_active":"2024-05-11T12:21:08.516316+0000","last_became_peered":"2024-05-11T12:21:08.516316+0000","last_unstale":"2024-05-11T12:24:03.871569+0000","last_undegraded":"2024-05-11T12:24:03.871569+0000","last_fullsized":"2024-05-11T12:24:03.871569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:25.769713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62307,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":325,"num_read_kb":223,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545930+0000","last_change":"2024-05-11T12:21:11.528553+0000","last_active":"2024-05-11T12:23:11.545930+0000","last_peered":"2024-05-11T12:23:11.545930+0000","last_clean":"2024-05-11T12:23:11.545930+0000","last_became_active":"2024-05-11T12:21:11.528175+0000","last_became_peered":"2024-05-11T12:21:11.528175+0000","last_unstale":"2024-05-11T12:23:11.545930+0000","last_undegraded":"2024-05-11T12:23:11.545930+0000","last_fullsized":"2024-05-11T12:23:11.545930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:46:42.689851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651107+0000","last_change":"2024-05-11T12:21:42.673360+0000","last_active":"2024-05-11T12:22:43.651107+0000","last_peered":"2024-05-11T12:22:43.651107+0000","last_clean":"2024-05-11T12:22:43.651107+0000","last_became_active":"2024-05-11T12:21:42.664842+0000","last_became_peered":"2024-05-11T12:21:42.664842+0000","last_unstale":"2024-05-11T12:22:43.651107+0000","last_undegraded":"2024-05-11T12:22:43.651107+0000","last_fullsized":"2024-05-11T12:22:43.651107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:40.271658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.552610+0000","last_change":"2024-05-11T12:21:11.533157+0000","last_active":"2024-05-11T12:23:11.552610+0000","last_peered":"2024-05-11T12:23:11.552610+0000","last_clean":"2024-05-11T12:23:11.552610+0000","last_became_active":"2024-05-11T12:21:11.532814+0000","last_became_peered":"2024-05-11T12:21:11.532814+0000","last_unstale":"2024-05-11T12:23:11.552610+0000","last_undegraded":"2024-05-11T12:23:11.552610+0000","last_fullsized":"2024-05-11T12:23:11.552610+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:30:56.555384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751604+0000","last_change":"2024-05-11T12:21:08.511907+0000","last_active":"2024-05-11T12:22:43.751604+0000","last_peered":"2024-05-11T12:22:43.751604+0000","last_clean":"2024-05-11T12:22:43.751604+0000","last_became_active":"2024-05-11T12:21:08.511296+0000","last_became_peered":"2024-05-11T12:21:08.511296+0000","last_unstale":"2024-05-11T12:22:43.751604+0000","last_undegraded":"2024-05-11T12:22:43.751604+0000","last_fullsized":"2024-05-11T12:22:43.751604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:43:17.374216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562158+0000","last_change":"2024-05-11T12:21:11.537859+0000","last_active":"2024-05-11T12:23:11.562158+0000","last_peered":"2024-05-11T12:23:11.562158+0000","last_clean":"2024-05-11T12:23:11.562158+0000","last_became_active":"2024-05-11T12:21:11.537448+0000","last_became_peered":"2024-05-11T12:21:11.537448+0000","last_unstale":"2024-05-11T12:23:11.562158+0000","last_undegraded":"2024-05-11T12:23:11.562158+0000","last_fullsized":"2024-05-11T12:23:11.562158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:09:33.866539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647911+0000","last_change":"2024-05-11T12:21:08.510228+0000","last_active":"2024-05-11T12:22:43.647911+0000","last_peered":"2024-05-11T12:22:43.647911+0000","last_clean":"2024-05-11T12:22:43.647911+0000","last_became_active":"2024-05-11T12:21:08.509671+0000","last_became_peered":"2024-05-11T12:21:08.509671+0000","last_unstale":"2024-05-11T12:22:43.647911+0000","last_undegraded":"2024-05-11T12:22:43.647911+0000","last_fullsized":"2024-05-11T12:22:43.647911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:06.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556927+0000","last_change":"2024-05-11T12:21:11.530935+0000","last_active":"2024-05-11T12:23:11.556927+0000","last_peered":"2024-05-11T12:23:11.556927+0000","last_clean":"2024-05-11T12:23:11.556927+0000","last_became_active":"2024-05-11T12:21:11.530579+0000","last_became_peered":"2024-05-11T12:21:11.530579+0000","last_unstale":"2024-05-11T12:23:11.556927+0000","last_undegraded":"2024-05-11T12:23:11.556927+0000","last_fullsized":"2024-05-11T12:23:11.556927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:38.296021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'8","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651091+0000","last_change":"2024-05-11T12:21:42.681114+0000","last_active":"2024-05-11T12:22:43.651091+0000","last_peered":"2024-05-11T12:22:43.651091+0000","last_clean":"2024-05-11T12:22:43.651091+0000","last_became_active":"2024-05-11T12:21:42.666878+0000","last_became_peered":"2024-05-11T12:21:42.666878+0000","last_unstale":"2024-05-11T12:22:43.651091+0000","last_undegraded":"2024-05-11T12:22:43.651091+0000","last_fullsized":"2024-05-11T12:22:43.651091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:45.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547859+0000","last_change":"2024-05-11T12:22:43.892989+0000","last_active":"2024-05-11T12:23:11.547859+0000","last_peered":"2024-05-11T12:23:11.547859+0000","last_clean":"2024-05-11T12:23:11.547859+0000","last_became_active":"2024-05-11T12:22:43.892599+0000","last_became_peered":"2024-05-11T12:22:43.892599+0000","last_unstale":"2024-05-11T12:23:11.547859+0000","last_undegraded":"2024-05-11T12:23:11.547859+0000","last_fullsized":"2024-05-11T12:23:11.547859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:46.357968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094221+0000","last_change":"2024-05-11T12:21:08.510504+0000","last_active":"2024-05-11T12:22:44.094221+0000","last_peered":"2024-05-11T12:22:44.094221+0000","last_clean":"2024-05-11T12:22:44.094221+0000","last_became_active":"2024-05-11T12:21:08.510074+0000","last_became_peered":"2024-05-11T12:21:08.510074+0000","last_unstale":"2024-05-11T12:22:44.094221+0000","last_undegraded":"2024-05-11T12:22:44.094221+0000","last_fullsized":"2024-05-11T12:22:44.094221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:31:49.465252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.555652+0000","last_change":"2024-05-11T12:22:43.753140+0000","last_active":"2024-05-11T12:23:11.555652+0000","last_peered":"2024-05-11T12:23:11.555652+0000","last_clean":"2024-05-11T12:23:11.555652+0000","last_became_active":"2024-05-11T12:22:43.752693+0000","last_became_peered":"2024-05-11T12:22:43.752693+0000","last_unstale":"2024-05-11T12:23:11.555652+0000","last_undegraded":"2024-05-11T12:23:11.555652+0000","last_fullsized":"2024-05-11T12:23:11.555652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:13.152017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647507+0000","last_change":"2024-05-11T12:21:08.531926+0000","last_active":"2024-05-11T12:22:43.647507+0000","last_peered":"2024-05-11T12:22:43.647507+0000","last_clean":"2024-05-11T12:22:43.647507+0000","last_became_active":"2024-05-11T12:21:08.531548+0000","last_became_peered":"2024-05-11T12:21:08.531548+0000","last_unstale":"2024-05-11T12:22:43.647507+0000","last_undegraded":"2024-05-11T12:22:43.647507+0000","last_fullsized":"2024-05-11T12:22:43.647507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:25:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548090+0000","last_change":"2024-05-11T12:21:11.528595+0000","last_active":"2024-05-11T12:23:11.548090+0000","last_peered":"2024-05-11T12:23:11.548090+0000","last_clean":"2024-05-11T12:23:11.548090+0000","last_became_active":"2024-05-11T12:21:11.527782+0000","last_became_peered":"2024-05-11T12:21:11.527782+0000","last_unstale":"2024-05-11T12:23:11.548090+0000","last_undegraded":"2024-05-11T12:23:11.548090+0000","last_fullsized":"2024-05-11T12:23:11.548090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:31:00.337497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649651+0000","last_change":"2024-05-11T12:21:08.508093+0000","last_active":"2024-05-11T12:22:43.649651+0000","last_peered":"2024-05-11T12:22:43.649651+0000","last_clean":"2024-05-11T12:22:43.649651+0000","last_became_active":"2024-05-11T12:21:08.507710+0000","last_became_peered":"2024-05-11T12:21:08.507710+0000","last_unstale":"2024-05-11T12:22:43.649651+0000","last_undegraded":"2024-05-11T12:22:43.649651+0000","last_fullsized":"2024-05-11T12:22:43.649651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:24.273932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.557566+0000","last_change":"2024-05-11T12:21:11.526026+0000","last_active":"2024-05-11T12:23:11.557566+0000","last_peered":"2024-05-11T12:23:11.557566+0000","last_clean":"2024-05-11T12:23:11.557566+0000","last_became_active":"2024-05-11T12:21:11.525628+0000","last_became_peered":"2024-05-11T12:21:11.525628+0000","last_unstale":"2024-05-11T12:23:11.557566+0000","last_undegraded":"2024-05-11T12:23:11.557566+0000","last_fullsized":"2024-05-11T12:23:11.557566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:12.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561475+0000","last_change":"2024-05-11T12:21:11.535808+0000","last_active":"2024-05-11T12:23:11.561475+0000","last_peered":"2024-05-11T12:23:11.561475+0000","last_clean":"2024-05-11T12:23:11.561475+0000","last_became_active":"2024-05-11T12:21:11.535204+0000","last_became_peered":"2024-05-11T12:21:11.535204+0000","last_unstale":"2024-05-11T12:23:11.561475+0000","last_undegraded":"2024-05-11T12:23:11.561475+0000","last_fullsized":"2024-05-11T12:23:11.561475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:36:01.208543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844644+0000","last_change":"2024-05-11T12:21:08.531838+0000","last_active":"2024-05-11T12:22:43.844644+0000","last_peered":"2024-05-11T12:22:43.844644+0000","last_clean":"2024-05-11T12:22:43.844644+0000","last_became_active":"2024-05-11T12:21:08.531491+0000","last_became_peered":"2024-05-11T12:21:08.531491+0000","last_unstale":"2024-05-11T12:22:43.844644+0000","last_undegraded":"2024-05-11T12:22:43.844644+0000","last_fullsized":"2024-05-11T12:22:43.844644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:33.928985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.535726+0000","last_change":"2024-05-11T12:21:11.538563+0000","last_active":"2024-05-11T12:23:11.535726+0000","last_peered":"2024-05-11T12:23:11.535726+0000","last_clean":"2024-05-11T12:23:11.535726+0000","last_became_active":"2024-05-11T12:21:11.538200+0000","last_became_peered":"2024-05-11T12:21:11.538200+0000","last_unstale":"2024-05-11T12:23:11.535726+0000","last_undegraded":"2024-05-11T12:23:11.535726+0000","last_fullsized":"2024-05-11T12:23:11.535726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:18.879528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648409+0000","last_change":"2024-05-11T12:21:08.509972+0000","last_active":"2024-05-11T12:22:43.648409+0000","last_peered":"2024-05-11T12:22:43.648409+0000","last_clean":"2024-05-11T12:22:43.648409+0000","last_became_active":"2024-05-11T12:21:08.509531+0000","last_became_peered":"2024-05-11T12:21:08.509531+0000","last_unstale":"2024-05-11T12:22:43.648409+0000","last_undegraded":"2024-05-11T12:22:43.648409+0000","last_fullsized":"2024-05-11T12:22:43.648409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:47:13.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649321+0000","last_change":"2024-05-11T12:19:42.546860+0000","last_active":"2024-05-11T12:22:43.649321+0000","last_peered":"2024-05-11T12:22:43.649321+0000","last_clean":"2024-05-11T12:22:43.649321+0000","last_became_active":"2024-05-11T12:19:42.541449+0000","last_became_peered":"2024-05-11T12:19:42.541449+0000","last_unstale":"2024-05-11T12:22:43.649321+0000","last_undegraded":"2024-05-11T12:22:43.649321+0000","last_fullsized":"2024-05-11T12:22:43.649321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_clean_scrub_stamp":"2024-05-11T12:16:12.381187+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:45.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559597+0000","last_change":"2024-05-11T12:21:11.536770+0000","last_active":"2024-05-11T12:23:11.559597+0000","last_peered":"2024-05-11T12:23:11.559597+0000","last_clean":"2024-05-11T12:23:11.559597+0000","last_became_active":"2024-05-11T12:21:11.536421+0000","last_became_peered":"2024-05-11T12:21:11.536421+0000","last_unstale":"2024-05-11T12:23:11.559597+0000","last_undegraded":"2024-05-11T12:23:11.559597+0000","last_fullsized":"2024-05-11T12:23:11.559597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:42.217398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649356+0000","last_change":"2024-05-11T12:21:42.675283+0000","last_active":"2024-05-11T12:22:43.649356+0000","last_peered":"2024-05-11T12:22:43.649356+0000","last_clean":"2024-05-11T12:22:43.649356+0000","last_became_active":"2024-05-11T12:21:42.674775+0000","last_became_peered":"2024-05-11T12:21:42.674775+0000","last_unstale":"2024-05-11T12:22:43.649356+0000","last_undegraded":"2024-05-11T12:22:43.649356+0000","last_fullsized":"2024-05-11T12:22:43.649356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:13:30.143988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094224+0000","last_change":"2024-05-11T12:21:08.533462+0000","last_active":"2024-05-11T12:22:44.094224+0000","last_peered":"2024-05-11T12:22:44.094224+0000","last_clean":"2024-05-11T12:22:44.094224+0000","last_became_active":"2024-05-11T12:21:08.533081+0000","last_became_peered":"2024-05-11T12:21:08.533081+0000","last_unstale":"2024-05-11T12:22:44.094224+0000","last_undegraded":"2024-05-11T12:22:44.094224+0000","last_fullsized":"2024-05-11T12:22:44.094224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:01:48.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.549932+0000","last_change":"2024-05-11T12:21:11.539484+0000","last_active":"2024-05-11T12:23:11.549932+0000","last_peered":"2024-05-11T12:23:11.549932+0000","last_clean":"2024-05-11T12:23:11.549932+0000","last_became_active":"2024-05-11T12:21:11.536810+0000","last_became_peered":"2024-05-11T12:21:11.536810+0000","last_unstale":"2024-05-11T12:23:11.549932+0000","last_undegraded":"2024-05-11T12:23:11.549932+0000","last_fullsized":"2024-05-11T12:23:11.549932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:47:41.461766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'11","reported_seq":42,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.614778+0000","last_change":"2024-05-11T12:21:08.510519+0000","last_active":"2024-05-11T12:23:11.614778+0000","last_peered":"2024-05-11T12:23:11.614778+0000","last_clean":"2024-05-11T12:23:11.614778+0000","last_became_active":"2024-05-11T12:21:08.510180+0000","last_became_peered":"2024-05-11T12:21:08.510180+0000","last_unstale":"2024-05-11T12:23:11.614778+0000","last_undegraded":"2024-05-11T12:23:11.614778+0000","last_fullsized":"2024-05-11T12:23:11.614778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:24.537043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553050+0000","last_change":"2024-05-11T12:21:11.533726+0000","last_active":"2024-05-11T12:23:11.553050+0000","last_peered":"2024-05-11T12:23:11.553050+0000","last_clean":"2024-05-11T12:23:11.553050+0000","last_became_active":"2024-05-11T12:21:11.533393+0000","last_became_peered":"2024-05-11T12:21:11.533393+0000","last_unstale":"2024-05-11T12:23:11.553050+0000","last_undegraded":"2024-05-11T12:23:11.553050+0000","last_fullsized":"2024-05-11T12:23:11.553050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:03.071666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648390+0000","last_change":"2024-05-11T12:21:09.036622+0000","last_active":"2024-05-11T12:22:43.648390+0000","last_peered":"2024-05-11T12:22:43.648390+0000","last_clean":"2024-05-11T12:22:43.648390+0000","last_became_active":"2024-05-11T12:21:09.036209+0000","last_became_peered":"2024-05-11T12:21:09.036209+0000","last_unstale":"2024-05-11T12:22:43.648390+0000","last_undegraded":"2024-05-11T12:22:43.648390+0000","last_fullsized":"2024-05-11T12:22:43.648390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:29:58.947588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553776+0000","last_change":"2024-05-11T12:22:43.846111+0000","last_active":"2024-05-11T12:23:11.553776+0000","last_peered":"2024-05-11T12:23:11.553776+0000","last_clean":"2024-05-11T12:23:11.553776+0000","last_became_active":"2024-05-11T12:22:43.845712+0000","last_became_peered":"2024-05-11T12:22:43.845712+0000","last_unstale":"2024-05-11T12:23:11.553776+0000","last_undegraded":"2024-05-11T12:23:11.553776+0000","last_fullsized":"2024-05-11T12:23:11.553776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:19:47.516917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891608+0000","last_change":"2024-05-11T12:21:08.533435+0000","last_active":"2024-05-11T12:22:43.891608+0000","last_peered":"2024-05-11T12:22:43.891608+0000","last_clean":"2024-05-11T12:22:43.891608+0000","last_became_active":"2024-05-11T12:21:08.533100+0000","last_became_peered":"2024-05-11T12:21:08.533100+0000","last_unstale":"2024-05-11T12:22:43.891608+0000","last_undegraded":"2024-05-11T12:22:43.891608+0000","last_fullsized":"2024-05-11T12:22:43.891608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:17.619015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.529373+0000","last_change":"2024-05-11T12:21:11.526140+0000","last_active":"2024-05-11T12:23:11.529373+0000","last_peered":"2024-05-11T12:23:11.529373+0000","last_clean":"2024-05-11T12:23:11.529373+0000","last_became_active":"2024-05-11T12:21:11.525810+0000","last_became_peered":"2024-05-11T12:21:11.525810+0000","last_unstale":"2024-05-11T12:23:11.529373+0000","last_undegraded":"2024-05-11T12:23:11.529373+0000","last_fullsized":"2024-05-11T12:23:11.529373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:14:04.160379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790116+0000","last_change":"2024-05-11T12:21:08.508031+0000","last_active":"2024-05-11T12:22:43.790116+0000","last_peered":"2024-05-11T12:22:43.790116+0000","last_clean":"2024-05-11T12:22:43.790116+0000","last_became_active":"2024-05-11T12:21:08.507703+0000","last_became_peered":"2024-05-11T12:21:08.507703+0000","last_unstale":"2024-05-11T12:22:43.790116+0000","last_undegraded":"2024-05-11T12:22:43.790116+0000","last_fullsized":"2024-05-11T12:22:43.790116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:05.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547615+0000","last_change":"2024-05-11T12:21:11.528293+0000","last_active":"2024-05-11T12:23:11.547615+0000","last_peered":"2024-05-11T12:23:11.547615+0000","last_clean":"2024-05-11T12:23:11.547615+0000","last_became_active":"2024-05-11T12:21:11.527862+0000","last_became_peered":"2024-05-11T12:21:11.527862+0000","last_unstale":"2024-05-11T12:23:11.547615+0000","last_undegraded":"2024-05-11T12:23:11.547615+0000","last_fullsized":"2024-05-11T12:23:11.547615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:07.335829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653717+0000","last_change":"2024-05-11T12:21:08.510138+0000","last_active":"2024-05-11T12:22:43.653717+0000","last_peered":"2024-05-11T12:22:43.653717+0000","last_clean":"2024-05-11T12:22:43.653717+0000","last_became_active":"2024-05-11T12:21:08.509566+0000","last_became_peered":"2024-05-11T12:21:08.509566+0000","last_unstale":"2024-05-11T12:22:43.653717+0000","last_undegraded":"2024-05-11T12:22:43.653717+0000","last_fullsized":"2024-05-11T12:22:43.653717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:24:58.174305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.542051+0000","last_change":"2024-05-11T12:22:43.750230+0000","last_active":"2024-05-11T12:23:11.542051+0000","last_peered":"2024-05-11T12:23:11.542051+0000","last_clean":"2024-05-11T12:23:11.542051+0000","last_became_active":"2024-05-11T12:22:43.749836+0000","last_became_peered":"2024-05-11T12:22:43.749836+0000","last_unstale":"2024-05-11T12:23:11.542051+0000","last_undegraded":"2024-05-11T12:23:11.542051+0000","last_fullsized":"2024-05-11T12:23:11.542051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:36.498862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"79'2","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093949+0000","last_change":"2024-05-11T12:21:42.683706+0000","last_active":"2024-05-11T12:22:44.093949+0000","last_peered":"2024-05-11T12:22:44.093949+0000","last_clean":"2024-05-11T12:22:44.093949+0000","last_became_active":"2024-05-11T12:21:42.675791+0000","last_became_peered":"2024-05-11T12:21:42.675791+0000","last_unstale":"2024-05-11T12:22:44.093949+0000","last_undegraded":"2024-05-11T12:22:44.093949+0000","last_fullsized":"2024-05-11T12:22:44.093949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:13:26.737525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4],"acting":[8,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560452+0000","last_change":"2024-05-11T12:21:11.544153+0000","last_active":"2024-05-11T12:23:11.560452+0000","last_peered":"2024-05-11T12:23:11.560452+0000","last_clean":"2024-05-11T12:23:11.560452+0000","last_became_active":"2024-05-11T12:21:11.543772+0000","last_became_peered":"2024-05-11T12:21:11.543772+0000","last_unstale":"2024-05-11T12:23:11.560452+0000","last_undegraded":"2024-05-11T12:23:11.560452+0000","last_fullsized":"2024-05-11T12:23:11.560452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:54:30.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653862+0000","last_change":"2024-05-11T12:21:09.036409+0000","last_active":"2024-05-11T12:22:43.653862+0000","last_peered":"2024-05-11T12:22:43.653862+0000","last_clean":"2024-05-11T12:22:43.653862+0000","last_became_active":"2024-05-11T12:21:09.036009+0000","last_became_peered":"2024-05-11T12:21:09.036009+0000","last_unstale":"2024-05-11T12:22:43.653862+0000","last_undegraded":"2024-05-11T12:22:43.653862+0000","last_fullsized":"2024-05-11T12:22:43.653862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:55:28.736718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561529+0000","last_change":"2024-05-11T12:21:11.542685+0000","last_active":"2024-05-11T12:23:11.561529+0000","last_peered":"2024-05-11T12:23:11.561529+0000","last_clean":"2024-05-11T12:23:11.561529+0000","last_became_active":"2024-05-11T12:21:11.542130+0000","last_became_peered":"2024-05-11T12:21:11.542130+0000","last_unstale":"2024-05-11T12:23:11.561529+0000","last_undegraded":"2024-05-11T12:23:11.561529+0000","last_fullsized":"2024-05-11T12:23:11.561529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:22.578809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751961+0000","last_change":"2024-05-11T12:21:08.510869+0000","last_active":"2024-05-11T12:22:43.751961+0000","last_peered":"2024-05-11T12:22:43.751961+0000","last_clean":"2024-05-11T12:22:43.751961+0000","last_became_active":"2024-05-11T12:21:08.510510+0000","last_became_peered":"2024-05-11T12:21:08.510510+0000","last_unstale":"2024-05-11T12:22:43.751961+0000","last_undegraded":"2024-05-11T12:22:43.751961+0000","last_fullsized":"2024-05-11T12:22:43.751961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:50.002313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556787+0000","last_change":"2024-05-11T12:21:11.538622+0000","last_active":"2024-05-11T12:23:11.556787+0000","last_peered":"2024-05-11T12:23:11.556787+0000","last_clean":"2024-05-11T12:23:11.556787+0000","last_became_active":"2024-05-11T12:21:11.538229+0000","last_became_peered":"2024-05-11T12:21:11.538229+0000","last_unstale":"2024-05-11T12:23:11.556787+0000","last_undegraded":"2024-05-11T12:23:11.556787+0000","last_fullsized":"2024-05-11T12:23:11.556787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:01:15.912428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.845101+0000","last_change":"2024-05-11T12:21:42.677421+0000","last_active":"2024-05-11T12:22:43.845101+0000","last_peered":"2024-05-11T12:22:43.845101+0000","last_clean":"2024-05-11T12:22:43.845101+0000","last_became_active":"2024-05-11T12:21:42.672695+0000","last_became_peered":"2024-05-11T12:21:42.672695+0000","last_unstale":"2024-05-11T12:22:43.845101+0000","last_undegraded":"2024-05-11T12:22:43.845101+0000","last_fullsized":"2024-05-11T12:22:43.845101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:30:27.606816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556993+0000","last_change":"2024-05-11T12:21:11.536149+0000","last_active":"2024-05-11T12:23:11.556993+0000","last_peered":"2024-05-11T12:23:11.556993+0000","last_clean":"2024-05-11T12:23:11.556993+0000","last_became_active":"2024-05-11T12:21:11.535763+0000","last_became_peered":"2024-05-11T12:21:11.535763+0000","last_unstale":"2024-05-11T12:23:11.556993+0000","last_undegraded":"2024-05-11T12:23:11.556993+0000","last_fullsized":"2024-05-11T12:23:11.556993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:46:15.246610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547785+0000","last_change":"2024-05-11T12:21:11.534318+0000","last_active":"2024-05-11T12:23:11.547785+0000","last_peered":"2024-05-11T12:23:11.547785+0000","last_clean":"2024-05-11T12:23:11.547785+0000","last_became_active":"2024-05-11T12:21:11.533949+0000","last_became_peered":"2024-05-11T12:21:11.533949+0000","last_unstale":"2024-05-11T12:23:11.547785+0000","last_undegraded":"2024-05-11T12:23:11.547785+0000","last_fullsized":"2024-05-11T12:23:11.547785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:41:42.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561905+0000","last_change":"2024-05-11T12:21:11.529276+0000","last_active":"2024-05-11T12:23:11.561905+0000","last_peered":"2024-05-11T12:23:11.561905+0000","last_clean":"2024-05-11T12:23:11.561905+0000","last_became_active":"2024-05-11T12:21:11.528857+0000","last_became_peered":"2024-05-11T12:21:11.528857+0000","last_unstale":"2024-05-11T12:23:11.561905+0000","last_undegraded":"2024-05-11T12:23:11.561905+0000","last_fullsized":"2024-05-11T12:23:11.561905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:54.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548623+0000","last_change":"2024-05-11T12:21:11.532342+0000","last_active":"2024-05-11T12:23:11.548623+0000","last_peered":"2024-05-11T12:23:11.548623+0000","last_clean":"2024-05-11T12:23:11.548623+0000","last_became_active":"2024-05-11T12:21:11.531949+0000","last_became_peered":"2024-05-11T12:21:11.531949+0000","last_unstale":"2024-05-11T12:23:11.548623+0000","last_undegraded":"2024-05-11T12:23:11.548623+0000","last_fullsized":"2024-05-11T12:23:11.548623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:23:00.010874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548470+0000","last_change":"2024-05-11T12:21:11.531547+0000","last_active":"2024-05-11T12:23:11.548470+0000","last_peered":"2024-05-11T12:23:11.548470+0000","last_clean":"2024-05-11T12:23:11.548470+0000","last_became_active":"2024-05-11T12:21:11.529702+0000","last_became_peered":"2024-05-11T12:21:11.529702+0000","last_unstale":"2024-05-11T12:23:11.548470+0000","last_undegraded":"2024-05-11T12:23:11.548470+0000","last_fullsized":"2024-05-11T12:23:11.548470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:13.857026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.554216+0000","last_change":"2024-05-11T12:21:11.531261+0000","last_active":"2024-05-11T12:23:11.554216+0000","last_peered":"2024-05-11T12:23:11.554216+0000","last_clean":"2024-05-11T12:23:11.554216+0000","last_became_active":"2024-05-11T12:21:11.530561+0000","last_became_peered":"2024-05-11T12:21:11.530561+0000","last_unstale":"2024-05-11T12:23:11.554216+0000","last_undegraded":"2024-05-11T12:23:11.554216+0000","last_fullsized":"2024-05-11T12:23:11.554216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:52:07.654417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.546445+0000","last_change":"2024-05-11T12:21:11.533255+0000","last_active":"2024-05-11T12:23:11.546445+0000","last_peered":"2024-05-11T12:23:11.546445+0000","last_clean":"2024-05-11T12:23:11.546445+0000","last_became_active":"2024-05-11T12:21:11.532909+0000","last_became_peered":"2024-05-11T12:21:11.532909+0000","last_unstale":"2024-05-11T12:23:11.546445+0000","last_undegraded":"2024-05-11T12:23:11.546445+0000","last_fullsized":"2024-05-11T12:23:11.546445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:07:39.501482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553493+0000","last_change":"2024-05-11T12:21:11.528856+0000","last_active":"2024-05-11T12:23:11.553493+0000","last_peered":"2024-05-11T12:23:11.553493+0000","last_clean":"2024-05-11T12:23:11.553493+0000","last_became_active":"2024-05-11T12:21:11.528230+0000","last_became_peered":"2024-05-11T12:21:11.528230+0000","last_unstale":"2024-05-11T12:23:11.553493+0000","last_undegraded":"2024-05-11T12:23:11.553493+0000","last_fullsized":"2024-05-11T12:23:11.553493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:40.429045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553955+0000","last_change":"2024-05-11T12:21:11.534606+0000","last_active":"2024-05-11T12:23:11.553955+0000","last_peered":"2024-05-11T12:23:11.553955+0000","last_clean":"2024-05-11T12:23:11.553955+0000","last_became_active":"2024-05-11T12:21:11.534193+0000","last_became_peered":"2024-05-11T12:21:11.534193+0000","last_unstale":"2024-05-11T12:23:11.553955+0000","last_undegraded":"2024-05-11T12:23:11.553955+0000","last_fullsized":"2024-05-11T12:23:11.553955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:00:12.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548067+0000","last_change":"2024-05-11T12:21:11.525959+0000","last_active":"2024-05-11T12:23:11.548067+0000","last_peered":"2024-05-11T12:23:11.548067+0000","last_clean":"2024-05-11T12:23:11.548067+0000","last_became_active":"2024-05-11T12:21:11.525129+0000","last_became_peered":"2024-05-11T12:21:11.525129+0000","last_unstale":"2024-05-11T12:23:11.548067+0000","last_undegraded":"2024-05-11T12:23:11.548067+0000","last_fullsized":"2024-05-11T12:23:11.548067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:28.828336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547068+0000","last_change":"2024-05-11T12:21:11.538591+0000","last_active":"2024-05-11T12:23:11.547068+0000","last_peered":"2024-05-11T12:23:11.547068+0000","last_clean":"2024-05-11T12:23:11.547068+0000","last_became_active":"2024-05-11T12:21:11.537855+0000","last_became_peered":"2024-05-11T12:21:11.537855+0000","last_unstale":"2024-05-11T12:23:11.547068+0000","last_undegraded":"2024-05-11T12:23:11.547068+0000","last_fullsized":"2024-05-11T12:23:11.547068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:52.625763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.563270+0000","last_change":"2024-05-11T12:21:11.541185+0000","last_active":"2024-05-11T12:23:11.563270+0000","last_peered":"2024-05-11T12:23:11.563270+0000","last_clean":"2024-05-11T12:23:11.563270+0000","last_became_active":"2024-05-11T12:21:11.540799+0000","last_became_peered":"2024-05-11T12:21:11.540799+0000","last_unstale":"2024-05-11T12:23:11.563270+0000","last_undegraded":"2024-05-11T12:23:11.563270+0000","last_fullsized":"2024-05-11T12:23:11.563270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:24.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.552798+0000","last_change":"2024-05-11T12:21:11.527811+0000","last_active":"2024-05-11T12:23:11.552798+0000","last_peered":"2024-05-11T12:23:11.552798+0000","last_clean":"2024-05-11T12:23:11.552798+0000","last_became_active":"2024-05-11T12:21:11.526603+0000","last_became_peered":"2024-05-11T12:21:11.526603+0000","last_unstale":"2024-05-11T12:23:11.552798+0000","last_undegraded":"2024-05-11T12:23:11.552798+0000","last_fullsized":"2024-05-11T12:23:11.552798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:43.833506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561468+0000","last_change":"2024-05-11T12:21:11.530824+0000","last_active":"2024-05-11T12:23:11.561468+0000","last_peered":"2024-05-11T12:23:11.561468+0000","last_clean":"2024-05-11T12:23:11.561468+0000","last_became_active":"2024-05-11T12:21:11.530486+0000","last_became_peered":"2024-05-11T12:21:11.530486+0000","last_unstale":"2024-05-11T12:23:11.561468+0000","last_undegraded":"2024-05-11T12:23:11.561468+0000","last_fullsized":"2024-05-11T12:23:11.561468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:47.054822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247249,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63105,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":331,"num_read_kb":229,"num_write":92,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":189315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":445299,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":1098251,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037822,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445847,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":445869,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430281,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":1036332,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445382,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507572,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":445355,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":507572,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":445603,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445287,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1035932,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:24:09.563 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-11T12:24:09.564 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-11T12:24:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 -- 172.21.15.47:0/137231677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390105e60 msgr2=0x7f5390108250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 --2- 172.21.15.47:0/137231677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390105e60 0x7f5390108250 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5384009900 tx=0x7f538402a410 comp rx=0 tx=0).stop 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 -- 172.21.15.47:0/137231677 shutdown_connections 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 --2- 172.21.15.47:0/137231677 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5390108790 0x7f539010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 --2- 172.21.15.47:0/137231677 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390105e60 0x7f5390108250 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 --2- 172.21.15.47:0/137231677 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5390103530 0x7f5390105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 -- 172.21.15.47:0/137231677 >> 172.21.15.47:0/137231677 conn(0x7f53900fcf20 msgr2=0x7f53900ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 -- 172.21.15.47:0/137231677 shutdown_connections 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.700+0000 7f5396112640 1 -- 172.21.15.47:0/137231677 wait complete. 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 Processor -- start 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 -- start start 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5390103530 0x7f5390198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5390105e60 0x7f5390199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390108790 0x7f53901a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f539010e5e0 con 0x7f5390108790 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f539010e460 con 0x7f5390103530 2024-05-11T12:24:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f5396112640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f539010e760 con 0x7f5390105e60 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f538ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390108790 0x7f53901a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f538ffff640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390108790 0x7f53901a0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52432/0 (socket says 172.21.15.47:52432) 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.701+0000 7f538ffff640 1 -- 172.21.15.47:0/1534381000 learned_addr learned my addr 172.21.15.47:0/1534381000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538effd640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5390105e60 0x7f5390199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538f7fe640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5390103530 0x7f5390198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538ffff640 1 -- 172.21.15.47:0/1534381000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5390105e60 msgr2=0x7f5390199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538ffff640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5390105e60 0x7f5390199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538ffff640 1 -- 172.21.15.47:0/1534381000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5390103530 msgr2=0x7f5390198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538ffff640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5390103530 0x7f5390198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538ffff640 1 -- 172.21.15.47:0/1534381000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53901a0910 con 0x7f5390108790 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538f7fe640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5390103530 0x7f5390198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538ffff640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390108790 0x7f53901a0270 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f538800e970 tx=0x7f538800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538cff9640 1 -- 172.21.15.47:0/1534381000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538800ccc0 con 0x7f5390108790 2024-05-11T12:24:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538cff9640 1 -- 172.21.15.47:0/1534381000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f538800ce60 con 0x7f5390108790 2024-05-11T12:24:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901a0c00 con 0x7f5390108790 2024-05-11T12:24:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f538cff9640 1 -- 172.21.15.47:0/1534381000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388010430 con 0x7f5390108790 2024-05-11T12:24:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.702+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a11a0 con 0x7f5390108790 2024-05-11T12:24:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.703+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53900fea10 con 0x7f5390108790 2024-05-11T12:24:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.704+0000 7f538cff9640 1 -- 172.21.15.47:0/1534381000 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5388004f90 con 0x7f5390108790 2024-05-11T12:24:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.705+0000 7f538cff9640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f53780b2870 0x7f53780b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.705+0000 7f538f7fe640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f53780b2870 0x7f53780b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.705+0000 7f538cff9640 1 -- 172.21.15.47:0/1534381000 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f53880e2810 con 0x7f5390108790 2024-05-11T12:24:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.706+0000 7f538f7fe640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f53780b2870 0x7f53780b4d30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f539019a1d0 tx=0x7f5380009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:09.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.710+0000 7f538cff9640 1 -- 172.21.15.47:0/1534381000 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53880aa400 con 0x7f5390108790 2024-05-11T12:24:09.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.898+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 --> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5390062ea0 con 0x7f53780b2870 2024-05-11T12:24:09.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.905+0000 7f538cff9640 1 -- 172.21.15.47:0/1534381000 <== mgr.14152 v2:172.21.15.47:6800/811043456 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258997 (secure 0 0 0) 0x7f5390062ea0 con 0x7f53780b2870 2024-05-11T12:24:09.908 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:09.910 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-05-11T12:24:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f53780b2870 msgr2=0x7f53780b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f53780b2870 0x7f53780b4d30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f539019a1d0 tx=0x7f5380009530 comp rx=0 tx=0).stop 2024-05-11T12:24:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390108790 msgr2=0x7f53901a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:09.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390108790 0x7f53901a0270 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f538800e970 tx=0x7f538800ed70 comp rx=0 tx=0).stop 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 shutdown_connections 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f53780b2870 0x7f53780b4d30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5390108790 0x7f53901a0270 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5390105e60 0x7f5390199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 --2- 172.21.15.47:0/1534381000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5390103530 0x7f5390198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 >> 172.21.15.47:0/1534381000 conn(0x7f53900fcf20 msgr2=0x7f5390103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 shutdown_connections 2024-05-11T12:24:09.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:09.908+0000 7f5396112640 1 -- 172.21.15.47:0/1534381000 wait complete. 2024-05-11T12:24:09.920 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-05-11T12:24:09.576445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653707,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":407,"num_read_kb":293,"num_write":219,"num_write_kb":2245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2951874,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":671,"ondisk_log_size":671,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393340,"kb_used_data":10908,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124548484,"statfs":{"total":1151940427776,"available":1151537647616,"internally_reserved":0,"allocated":11169792,"data_stored":7304301,"data_compressed":246459,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":19142,"internal_metadata":391492922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002911"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553592+0000","last_change":"2024-05-11T12:21:11.536657+0000","last_active":"2024-05-11T12:23:11.553592+0000","last_peered":"2024-05-11T12:23:11.553592+0000","last_clean":"2024-05-11T12:23:11.553592+0000","last_became_active":"2024-05-11T12:21:11.536243+0000","last_became_peered":"2024-05-11T12:21:11.536243+0000","last_unstale":"2024-05-11T12:23:11.553592+0000","last_undegraded":"2024-05-11T12:23:11.553592+0000","last_fullsized":"2024-05-11T12:23:11.553592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:29:20.869387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545144+0000","last_change":"2024-05-11T12:21:11.527602+0000","last_active":"2024-05-11T12:23:11.545144+0000","last_peered":"2024-05-11T12:23:11.545144+0000","last_clean":"2024-05-11T12:23:11.545144+0000","last_became_active":"2024-05-11T12:21:11.527309+0000","last_became_peered":"2024-05-11T12:21:11.527309+0000","last_unstale":"2024-05-11T12:23:11.545144+0000","last_undegraded":"2024-05-11T12:23:11.545144+0000","last_fullsized":"2024-05-11T12:23:11.545144+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:18:03.764570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556325+0000","last_change":"2024-05-11T12:21:11.529094+0000","last_active":"2024-05-11T12:23:11.556325+0000","last_peered":"2024-05-11T12:23:11.556325+0000","last_clean":"2024-05-11T12:23:11.556325+0000","last_became_active":"2024-05-11T12:21:11.528796+0000","last_became_peered":"2024-05-11T12:21:11.528796+0000","last_unstale":"2024-05-11T12:23:11.556325+0000","last_undegraded":"2024-05-11T12:23:11.556325+0000","last_fullsized":"2024-05-11T12:23:11.556325+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:46:41.098066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561517+0000","last_change":"2024-05-11T12:21:11.528903+0000","last_active":"2024-05-11T12:23:11.561517+0000","last_peered":"2024-05-11T12:23:11.561517+0000","last_clean":"2024-05-11T12:23:11.561517+0000","last_became_active":"2024-05-11T12:21:11.528545+0000","last_became_peered":"2024-05-11T12:21:11.528545+0000","last_unstale":"2024-05-11T12:23:11.561517+0000","last_undegraded":"2024-05-11T12:23:11.561517+0000","last_fullsized":"2024-05-11T12:23:11.561517+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-13T00:03:56.424753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544651+0000","last_change":"2024-05-11T12:21:11.521074+0000","last_active":"2024-05-11T12:23:11.544651+0000","last_peered":"2024-05-11T12:23:11.544651+0000","last_clean":"2024-05-11T12:23:11.544651+0000","last_became_active":"2024-05-11T12:21:11.520675+0000","last_became_peered":"2024-05-11T12:21:11.520675+0000","last_unstale":"2024-05-11T12:23:11.544651+0000","last_undegraded":"2024-05-11T12:23:11.544651+0000","last_fullsized":"2024-05-11T12:23:11.544651+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:22:50.952687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547969+0000","last_change":"2024-05-11T12:21:42.671243+0000","last_active":"2024-05-11T12:23:11.547969+0000","last_peered":"2024-05-11T12:23:11.547969+0000","last_clean":"2024-05-11T12:23:11.547969+0000","last_became_active":"2024-05-11T12:21:42.670801+0000","last_became_peered":"2024-05-11T12:21:42.670801+0000","last_unstale":"2024-05-11T12:23:11.547969+0000","last_undegraded":"2024-05-11T12:23:11.547969+0000","last_fullsized":"2024-05-11T12:23:11.547969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:57:02.539855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,6],"acting":[10,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540731+0000","last_change":"2024-05-11T12:22:44.095706+0000","last_active":"2024-05-11T12:23:11.540731+0000","last_peered":"2024-05-11T12:23:11.540731+0000","last_clean":"2024-05-11T12:23:11.540731+0000","last_became_active":"2024-05-11T12:22:44.095177+0000","last_became_peered":"2024-05-11T12:22:44.095177+0000","last_unstale":"2024-05-11T12:23:11.540731+0000","last_undegraded":"2024-05-11T12:23:11.540731+0000","last_fullsized":"2024-05-11T12:23:11.540731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:11.426607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545528+0000","last_change":"2024-05-11T12:21:11.532874+0000","last_active":"2024-05-11T12:23:11.545528+0000","last_peered":"2024-05-11T12:23:11.545528+0000","last_clean":"2024-05-11T12:23:11.545528+0000","last_became_active":"2024-05-11T12:21:11.532237+0000","last_became_peered":"2024-05-11T12:21:11.532237+0000","last_unstale":"2024-05-11T12:23:11.545528+0000","last_undegraded":"2024-05-11T12:23:11.545528+0000","last_fullsized":"2024-05-11T12:23:11.545528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:13.770719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.557701+0000","last_change":"2024-05-11T12:21:11.536465+0000","last_active":"2024-05-11T12:23:11.557701+0000","last_peered":"2024-05-11T12:23:11.557701+0000","last_clean":"2024-05-11T12:23:11.557701+0000","last_became_active":"2024-05-11T12:21:11.536057+0000","last_became_peered":"2024-05-11T12:21:11.536057+0000","last_unstale":"2024-05-11T12:23:11.557701+0000","last_undegraded":"2024-05-11T12:23:11.557701+0000","last_fullsized":"2024-05-11T12:23:11.557701+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:22:45.005475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560592+0000","last_change":"2024-05-11T12:21:11.528187+0000","last_active":"2024-05-11T12:23:11.560592+0000","last_peered":"2024-05-11T12:23:11.560592+0000","last_clean":"2024-05-11T12:23:11.560592+0000","last_became_active":"2024-05-11T12:21:11.527780+0000","last_became_peered":"2024-05-11T12:21:11.527780+0000","last_unstale":"2024-05-11T12:23:11.560592+0000","last_undegraded":"2024-05-11T12:23:11.560592+0000","last_fullsized":"2024-05-11T12:23:11.560592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:48:23.484372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560911+0000","last_change":"2024-05-11T12:21:11.543388+0000","last_active":"2024-05-11T12:23:11.560911+0000","last_peered":"2024-05-11T12:23:11.560911+0000","last_clean":"2024-05-11T12:23:11.560911+0000","last_became_active":"2024-05-11T12:21:11.542984+0000","last_became_peered":"2024-05-11T12:21:11.542984+0000","last_unstale":"2024-05-11T12:23:11.560911+0000","last_undegraded":"2024-05-11T12:23:11.560911+0000","last_fullsized":"2024-05-11T12:23:11.560911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:25:52.122477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561371+0000","last_change":"2024-05-11T12:21:11.523355+0000","last_active":"2024-05-11T12:23:11.561371+0000","last_peered":"2024-05-11T12:23:11.561371+0000","last_clean":"2024-05-11T12:23:11.561371+0000","last_became_active":"2024-05-11T12:21:11.523007+0000","last_became_peered":"2024-05-11T12:21:11.523007+0000","last_unstale":"2024-05-11T12:23:11.561371+0000","last_undegraded":"2024-05-11T12:23:11.561371+0000","last_fullsized":"2024-05-11T12:23:11.561371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:29:57.197028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548605+0000","last_change":"2024-05-11T12:21:11.528072+0000","last_active":"2024-05-11T12:23:11.548605+0000","last_peered":"2024-05-11T12:23:11.548605+0000","last_clean":"2024-05-11T12:23:11.548605+0000","last_became_active":"2024-05-11T12:21:11.527478+0000","last_became_peered":"2024-05-11T12:21:11.527478+0000","last_unstale":"2024-05-11T12:23:11.548605+0000","last_undegraded":"2024-05-11T12:23:11.548605+0000","last_fullsized":"2024-05-11T12:23:11.548605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:57:29.001602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544905+0000","last_change":"2024-05-11T12:21:11.526862+0000","last_active":"2024-05-11T12:23:11.544905+0000","last_peered":"2024-05-11T12:23:11.544905+0000","last_clean":"2024-05-11T12:23:11.544905+0000","last_became_active":"2024-05-11T12:21:11.526553+0000","last_became_peered":"2024-05-11T12:21:11.526553+0000","last_unstale":"2024-05-11T12:23:11.544905+0000","last_undegraded":"2024-05-11T12:23:11.544905+0000","last_fullsized":"2024-05-11T12:23:11.544905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:02:48.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540462+0000","last_change":"2024-05-11T12:21:42.669393+0000","last_active":"2024-05-11T12:23:11.540462+0000","last_peered":"2024-05-11T12:23:11.540462+0000","last_clean":"2024-05-11T12:23:11.540462+0000","last_became_active":"2024-05-11T12:21:42.669000+0000","last_became_peered":"2024-05-11T12:21:42.669000+0000","last_unstale":"2024-05-11T12:23:11.540462+0000","last_undegraded":"2024-05-11T12:23:11.540462+0000","last_fullsized":"2024-05-11T12:23:11.540462+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:52:29.439588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,2],"acting":[0,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.563015+0000","last_change":"2024-05-11T12:21:11.540698+0000","last_active":"2024-05-11T12:23:11.563015+0000","last_peered":"2024-05-11T12:23:11.563015+0000","last_clean":"2024-05-11T12:23:11.563015+0000","last_became_active":"2024-05-11T12:21:11.539899+0000","last_became_peered":"2024-05-11T12:21:11.539899+0000","last_unstale":"2024-05-11T12:23:11.563015+0000","last_undegraded":"2024-05-11T12:23:11.563015+0000","last_fullsized":"2024-05-11T12:23:11.563015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:48.222664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553880+0000","last_change":"2024-05-11T12:21:11.537055+0000","last_active":"2024-05-11T12:23:11.553880+0000","last_peered":"2024-05-11T12:23:11.553880+0000","last_clean":"2024-05-11T12:23:11.553880+0000","last_became_active":"2024-05-11T12:21:11.536625+0000","last_became_peered":"2024-05-11T12:21:11.536625+0000","last_unstale":"2024-05-11T12:23:11.553880+0000","last_undegraded":"2024-05-11T12:23:11.553880+0000","last_fullsized":"2024-05-11T12:23:11.553880+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:33.453467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548280+0000","last_change":"2024-05-11T12:21:11.530254+0000","last_active":"2024-05-11T12:23:11.548280+0000","last_peered":"2024-05-11T12:23:11.548280+0000","last_clean":"2024-05-11T12:23:11.548280+0000","last_became_active":"2024-05-11T12:21:11.529890+0000","last_became_peered":"2024-05-11T12:21:11.529890+0000","last_unstale":"2024-05-11T12:23:11.548280+0000","last_undegraded":"2024-05-11T12:23:11.548280+0000","last_fullsized":"2024-05-11T12:23:11.548280+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:17:36.100931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.549156+0000","last_change":"2024-05-11T12:21:11.532893+0000","last_active":"2024-05-11T12:23:11.549156+0000","last_peered":"2024-05-11T12:23:11.549156+0000","last_clean":"2024-05-11T12:23:11.549156+0000","last_became_active":"2024-05-11T12:21:11.532539+0000","last_became_peered":"2024-05-11T12:21:11.532539+0000","last_unstale":"2024-05-11T12:23:11.549156+0000","last_undegraded":"2024-05-11T12:23:11.549156+0000","last_fullsized":"2024-05-11T12:23:11.549156+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:14:57.888102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"79'4","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647560+0000","last_change":"2024-05-11T12:21:08.510909+0000","last_active":"2024-05-11T12:22:43.647560+0000","last_peered":"2024-05-11T12:22:43.647560+0000","last_clean":"2024-05-11T12:22:43.647560+0000","last_became_active":"2024-05-11T12:21:08.509986+0000","last_became_peered":"2024-05-11T12:21:08.509986+0000","last_unstale":"2024-05-11T12:22:43.647560+0000","last_undegraded":"2024-05-11T12:22:43.647560+0000","last_fullsized":"2024-05-11T12:22:43.647560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:03:36.430087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547054+0000","last_change":"2024-05-11T12:21:11.532403+0000","last_active":"2024-05-11T12:23:11.547054+0000","last_peered":"2024-05-11T12:23:11.547054+0000","last_clean":"2024-05-11T12:23:11.547054+0000","last_became_active":"2024-05-11T12:21:11.531907+0000","last_became_peered":"2024-05-11T12:21:11.531907+0000","last_unstale":"2024-05-11T12:23:11.547054+0000","last_undegraded":"2024-05-11T12:23:11.547054+0000","last_fullsized":"2024-05-11T12:23:11.547054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:53:27.163229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790479+0000","last_change":"2024-05-11T12:21:08.531664+0000","last_active":"2024-05-11T12:22:43.790479+0000","last_peered":"2024-05-11T12:22:43.790479+0000","last_clean":"2024-05-11T12:22:43.790479+0000","last_became_active":"2024-05-11T12:21:08.531319+0000","last_became_peered":"2024-05-11T12:21:08.531319+0000","last_unstale":"2024-05-11T12:22:43.790479+0000","last_undegraded":"2024-05-11T12:22:43.790479+0000","last_fullsized":"2024-05-11T12:22:43.790479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:16:08.395483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556428+0000","last_change":"2024-05-11T12:21:11.532132+0000","last_active":"2024-05-11T12:23:11.556428+0000","last_peered":"2024-05-11T12:23:11.556428+0000","last_clean":"2024-05-11T12:23:11.556428+0000","last_became_active":"2024-05-11T12:21:11.529580+0000","last_became_peered":"2024-05-11T12:21:11.529580+0000","last_unstale":"2024-05-11T12:23:11.556428+0000","last_undegraded":"2024-05-11T12:23:11.556428+0000","last_fullsized":"2024-05-11T12:23:11.556428+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:42:29.599701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094372+0000","last_change":"2024-05-11T12:21:42.674898+0000","last_active":"2024-05-11T12:22:44.094372+0000","last_peered":"2024-05-11T12:22:44.094372+0000","last_clean":"2024-05-11T12:22:44.094372+0000","last_became_active":"2024-05-11T12:21:42.669853+0000","last_became_peered":"2024-05-11T12:21:42.669853+0000","last_unstale":"2024-05-11T12:22:44.094372+0000","last_undegraded":"2024-05-11T12:22:44.094372+0000","last_fullsized":"2024-05-11T12:22:44.094372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:56:36.040786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559948+0000","last_change":"2024-05-11T12:21:11.534706+0000","last_active":"2024-05-11T12:23:11.559948+0000","last_peered":"2024-05-11T12:23:11.559948+0000","last_clean":"2024-05-11T12:23:11.559948+0000","last_became_active":"2024-05-11T12:21:11.534301+0000","last_became_peered":"2024-05-11T12:21:11.534301+0000","last_unstale":"2024-05-11T12:23:11.559948+0000","last_undegraded":"2024-05-11T12:23:11.559948+0000","last_fullsized":"2024-05-11T12:23:11.559948+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:21:24.277988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094123+0000","last_change":"2024-05-11T12:21:08.513794+0000","last_active":"2024-05-11T12:22:44.094123+0000","last_peered":"2024-05-11T12:22:44.094123+0000","last_clean":"2024-05-11T12:22:44.094123+0000","last_became_active":"2024-05-11T12:21:08.513383+0000","last_became_peered":"2024-05-11T12:21:08.513383+0000","last_unstale":"2024-05-11T12:22:44.094123+0000","last_undegraded":"2024-05-11T12:22:44.094123+0000","last_fullsized":"2024-05-11T12:22:44.094123+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:56:50.558886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562588+0000","last_change":"2024-05-11T12:21:11.537147+0000","last_active":"2024-05-11T12:23:11.562588+0000","last_peered":"2024-05-11T12:23:11.562588+0000","last_clean":"2024-05-11T12:23:11.562588+0000","last_became_active":"2024-05-11T12:21:11.536739+0000","last_became_peered":"2024-05-11T12:21:11.536739+0000","last_unstale":"2024-05-11T12:23:11.562588+0000","last_undegraded":"2024-05-11T12:23:11.562588+0000","last_fullsized":"2024-05-11T12:23:11.562588+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:00:45.556863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751625+0000","last_change":"2024-05-11T12:21:08.513494+0000","last_active":"2024-05-11T12:22:43.751625+0000","last_peered":"2024-05-11T12:22:43.751625+0000","last_clean":"2024-05-11T12:22:43.751625+0000","last_became_active":"2024-05-11T12:21:08.513089+0000","last_became_peered":"2024-05-11T12:21:08.513089+0000","last_unstale":"2024-05-11T12:22:43.751625+0000","last_undegraded":"2024-05-11T12:22:43.751625+0000","last_fullsized":"2024-05-11T12:22:43.751625+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:48:28.587032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.536678+0000","last_change":"2024-05-11T12:21:11.527827+0000","last_active":"2024-05-11T12:23:11.536678+0000","last_peered":"2024-05-11T12:23:11.536678+0000","last_clean":"2024-05-11T12:23:11.536678+0000","last_became_active":"2024-05-11T12:21:11.527397+0000","last_became_peered":"2024-05-11T12:21:11.527397+0000","last_unstale":"2024-05-11T12:23:11.536678+0000","last_undegraded":"2024-05-11T12:23:11.536678+0000","last_fullsized":"2024-05-11T12:23:11.536678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:18.710855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"79'1","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.748805+0000","last_change":"2024-05-11T12:21:08.513519+0000","last_active":"2024-05-11T12:22:43.748805+0000","last_peered":"2024-05-11T12:22:43.748805+0000","last_clean":"2024-05-11T12:22:43.748805+0000","last_became_active":"2024-05-11T12:21:08.512566+0000","last_became_peered":"2024-05-11T12:21:08.512566+0000","last_unstale":"2024-05-11T12:22:43.748805+0000","last_undegraded":"2024-05-11T12:22:43.748805+0000","last_fullsized":"2024-05-11T12:22:43.748805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:54:56.104043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562028+0000","last_change":"2024-05-11T12:21:11.535364+0000","last_active":"2024-05-11T12:23:11.562028+0000","last_peered":"2024-05-11T12:23:11.562028+0000","last_clean":"2024-05-11T12:23:11.562028+0000","last_became_active":"2024-05-11T12:21:11.534988+0000","last_became_peered":"2024-05-11T12:21:11.534988+0000","last_unstale":"2024-05-11T12:23:11.562028+0000","last_undegraded":"2024-05-11T12:23:11.562028+0000","last_fullsized":"2024-05-11T12:23:11.562028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:25:20.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844753+0000","last_change":"2024-05-11T12:21:08.509913+0000","last_active":"2024-05-11T12:22:43.844753+0000","last_peered":"2024-05-11T12:22:43.844753+0000","last_clean":"2024-05-11T12:22:43.844753+0000","last_became_active":"2024-05-11T12:21:08.509498+0000","last_became_peered":"2024-05-11T12:21:08.509498+0000","last_unstale":"2024-05-11T12:22:43.844753+0000","last_undegraded":"2024-05-11T12:22:43.844753+0000","last_fullsized":"2024-05-11T12:22:43.844753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:04:43.282700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.540114+0000","last_change":"2024-05-11T12:21:11.533108+0000","last_active":"2024-05-11T12:23:11.540114+0000","last_peered":"2024-05-11T12:23:11.540114+0000","last_clean":"2024-05-11T12:23:11.540114+0000","last_became_active":"2024-05-11T12:21:11.532779+0000","last_became_peered":"2024-05-11T12:21:11.532779+0000","last_unstale":"2024-05-11T12:23:11.540114+0000","last_undegraded":"2024-05-11T12:23:11.540114+0000","last_fullsized":"2024-05-11T12:23:11.540114+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:14:55.311786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751328+0000","last_change":"2024-05-11T12:21:08.513355+0000","last_active":"2024-05-11T12:22:43.751328+0000","last_peered":"2024-05-11T12:22:43.751328+0000","last_clean":"2024-05-11T12:22:43.751328+0000","last_became_active":"2024-05-11T12:21:08.512964+0000","last_became_peered":"2024-05-11T12:21:08.512964+0000","last_unstale":"2024-05-11T12:22:43.751328+0000","last_undegraded":"2024-05-11T12:22:43.751328+0000","last_fullsized":"2024-05-11T12:22:43.751328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:06:12.296865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548024+0000","last_change":"2024-05-11T12:21:11.525958+0000","last_active":"2024-05-11T12:23:11.548024+0000","last_peered":"2024-05-11T12:23:11.548024+0000","last_clean":"2024-05-11T12:23:11.548024+0000","last_became_active":"2024-05-11T12:21:11.525285+0000","last_became_peered":"2024-05-11T12:21:11.525285+0000","last_unstale":"2024-05-11T12:23:11.548024+0000","last_undegraded":"2024-05-11T12:23:11.548024+0000","last_fullsized":"2024-05-11T12:23:11.548024+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:24:47.262204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.749227+0000","last_change":"2024-05-11T12:21:42.673378+0000","last_active":"2024-05-11T12:22:43.749227+0000","last_peered":"2024-05-11T12:22:43.749227+0000","last_clean":"2024-05-11T12:22:43.749227+0000","last_became_active":"2024-05-11T12:21:42.672870+0000","last_became_peered":"2024-05-11T12:21:42.672870+0000","last_unstale":"2024-05-11T12:22:43.749227+0000","last_undegraded":"2024-05-11T12:22:43.749227+0000","last_fullsized":"2024-05-11T12:22:43.749227+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:55:41.845162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,9],"acting":[10,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559484+0000","last_change":"2024-05-11T12:21:11.538946+0000","last_active":"2024-05-11T12:23:11.559484+0000","last_peered":"2024-05-11T12:23:11.559484+0000","last_clean":"2024-05-11T12:23:11.559484+0000","last_became_active":"2024-05-11T12:21:11.535804+0000","last_became_peered":"2024-05-11T12:21:11.535804+0000","last_unstale":"2024-05-11T12:23:11.559484+0000","last_undegraded":"2024-05-11T12:23:11.559484+0000","last_fullsized":"2024-05-11T12:23:11.559484+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:37:16.221144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649463+0000","last_change":"2024-05-11T12:21:42.674203+0000","last_active":"2024-05-11T12:22:43.649463+0000","last_peered":"2024-05-11T12:22:43.649463+0000","last_clean":"2024-05-11T12:22:43.649463+0000","last_became_active":"2024-05-11T12:21:42.673478+0000","last_became_peered":"2024-05-11T12:21:42.673478+0000","last_unstale":"2024-05-11T12:22:43.649463+0000","last_undegraded":"2024-05-11T12:22:43.649463+0000","last_fullsized":"2024-05-11T12:22:43.649463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:26:20.648528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553883+0000","last_change":"2024-05-11T12:21:11.535346+0000","last_active":"2024-05-11T12:23:11.553883+0000","last_peered":"2024-05-11T12:23:11.553883+0000","last_clean":"2024-05-11T12:23:11.553883+0000","last_became_active":"2024-05-11T12:21:11.530262+0000","last_became_peered":"2024-05-11T12:21:11.530262+0000","last_unstale":"2024-05-11T12:23:11.553883+0000","last_undegraded":"2024-05-11T12:23:11.553883+0000","last_fullsized":"2024-05-11T12:23:11.553883+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:22:13.169797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"79'3","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.752106+0000","last_change":"2024-05-11T12:21:08.531598+0000","last_active":"2024-05-11T12:22:43.752106+0000","last_peered":"2024-05-11T12:22:43.752106+0000","last_clean":"2024-05-11T12:22:43.752106+0000","last_became_active":"2024-05-11T12:21:08.531158+0000","last_became_peered":"2024-05-11T12:21:08.531158+0000","last_unstale":"2024-05-11T12:22:43.752106+0000","last_undegraded":"2024-05-11T12:22:43.752106+0000","last_fullsized":"2024-05-11T12:22:43.752106+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:17:43.528151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.544607+0000","last_change":"2024-05-11T12:22:44.094901+0000","last_active":"2024-05-11T12:23:11.544607+0000","last_peered":"2024-05-11T12:23:11.544607+0000","last_clean":"2024-05-11T12:23:11.544607+0000","last_became_active":"2024-05-11T12:22:44.094563+0000","last_became_peered":"2024-05-11T12:22:44.094563+0000","last_unstale":"2024-05-11T12:23:11.544607+0000","last_undegraded":"2024-05-11T12:23:11.544607+0000","last_fullsized":"2024-05-11T12:23:11.544607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:42:41.499957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,4],"acting":[9,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"83'33","reported_seq":389,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:24:03.871569+0000","last_change":"2024-05-11T12:21:08.516737+0000","last_active":"2024-05-11T12:24:03.871569+0000","last_peered":"2024-05-11T12:24:03.871569+0000","last_clean":"2024-05-11T12:24:03.871569+0000","last_became_active":"2024-05-11T12:21:08.516316+0000","last_became_peered":"2024-05-11T12:21:08.516316+0000","last_unstale":"2024-05-11T12:24:03.871569+0000","last_undegraded":"2024-05-11T12:24:03.871569+0000","last_fullsized":"2024-05-11T12:24:03.871569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:36:25.769713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62307,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":325,"num_read_kb":223,"num_write":33,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.545930+0000","last_change":"2024-05-11T12:21:11.528553+0000","last_active":"2024-05-11T12:23:11.545930+0000","last_peered":"2024-05-11T12:23:11.545930+0000","last_clean":"2024-05-11T12:23:11.545930+0000","last_became_active":"2024-05-11T12:21:11.528175+0000","last_became_peered":"2024-05-11T12:21:11.528175+0000","last_unstale":"2024-05-11T12:23:11.545930+0000","last_undegraded":"2024-05-11T12:23:11.545930+0000","last_fullsized":"2024-05-11T12:23:11.545930+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:46:42.689851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"79'2","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651107+0000","last_change":"2024-05-11T12:21:42.673360+0000","last_active":"2024-05-11T12:22:43.651107+0000","last_peered":"2024-05-11T12:22:43.651107+0000","last_clean":"2024-05-11T12:22:43.651107+0000","last_became_active":"2024-05-11T12:21:42.664842+0000","last_became_peered":"2024-05-11T12:21:42.664842+0000","last_unstale":"2024-05-11T12:22:43.651107+0000","last_undegraded":"2024-05-11T12:22:43.651107+0000","last_fullsized":"2024-05-11T12:22:43.651107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:40.271658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.552610+0000","last_change":"2024-05-11T12:21:11.533157+0000","last_active":"2024-05-11T12:23:11.552610+0000","last_peered":"2024-05-11T12:23:11.552610+0000","last_clean":"2024-05-11T12:23:11.552610+0000","last_became_active":"2024-05-11T12:21:11.532814+0000","last_became_peered":"2024-05-11T12:21:11.532814+0000","last_unstale":"2024-05-11T12:23:11.552610+0000","last_undegraded":"2024-05-11T12:23:11.552610+0000","last_fullsized":"2024-05-11T12:23:11.552610+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:30:56.555384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751604+0000","last_change":"2024-05-11T12:21:08.511907+0000","last_active":"2024-05-11T12:22:43.751604+0000","last_peered":"2024-05-11T12:22:43.751604+0000","last_clean":"2024-05-11T12:22:43.751604+0000","last_became_active":"2024-05-11T12:21:08.511296+0000","last_became_peered":"2024-05-11T12:21:08.511296+0000","last_unstale":"2024-05-11T12:22:43.751604+0000","last_undegraded":"2024-05-11T12:22:43.751604+0000","last_fullsized":"2024-05-11T12:22:43.751604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:43:17.374216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.562158+0000","last_change":"2024-05-11T12:21:11.537859+0000","last_active":"2024-05-11T12:23:11.562158+0000","last_peered":"2024-05-11T12:23:11.562158+0000","last_clean":"2024-05-11T12:23:11.562158+0000","last_became_active":"2024-05-11T12:21:11.537448+0000","last_became_peered":"2024-05-11T12:21:11.537448+0000","last_unstale":"2024-05-11T12:23:11.562158+0000","last_undegraded":"2024-05-11T12:23:11.562158+0000","last_fullsized":"2024-05-11T12:23:11.562158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:09:33.866539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647911+0000","last_change":"2024-05-11T12:21:08.510228+0000","last_active":"2024-05-11T12:22:43.647911+0000","last_peered":"2024-05-11T12:22:43.647911+0000","last_clean":"2024-05-11T12:22:43.647911+0000","last_became_active":"2024-05-11T12:21:08.509671+0000","last_became_peered":"2024-05-11T12:21:08.509671+0000","last_unstale":"2024-05-11T12:22:43.647911+0000","last_undegraded":"2024-05-11T12:22:43.647911+0000","last_fullsized":"2024-05-11T12:22:43.647911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:13:06.899971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556927+0000","last_change":"2024-05-11T12:21:11.530935+0000","last_active":"2024-05-11T12:23:11.556927+0000","last_peered":"2024-05-11T12:23:11.556927+0000","last_clean":"2024-05-11T12:23:11.556927+0000","last_became_active":"2024-05-11T12:21:11.530579+0000","last_became_peered":"2024-05-11T12:21:11.530579+0000","last_unstale":"2024-05-11T12:23:11.556927+0000","last_undegraded":"2024-05-11T12:23:11.556927+0000","last_fullsized":"2024-05-11T12:23:11.556927+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:38.296021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"79'8","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.651091+0000","last_change":"2024-05-11T12:21:42.681114+0000","last_active":"2024-05-11T12:22:43.651091+0000","last_peered":"2024-05-11T12:22:43.651091+0000","last_clean":"2024-05-11T12:22:43.651091+0000","last_became_active":"2024-05-11T12:21:42.666878+0000","last_became_peered":"2024-05-11T12:21:42.666878+0000","last_unstale":"2024-05-11T12:22:43.651091+0000","last_undegraded":"2024-05-11T12:22:43.651091+0000","last_fullsized":"2024-05-11T12:22:43.651091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:27:45.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547859+0000","last_change":"2024-05-11T12:22:43.892989+0000","last_active":"2024-05-11T12:23:11.547859+0000","last_peered":"2024-05-11T12:23:11.547859+0000","last_clean":"2024-05-11T12:23:11.547859+0000","last_became_active":"2024-05-11T12:22:43.892599+0000","last_became_peered":"2024-05-11T12:22:43.892599+0000","last_unstale":"2024-05-11T12:23:11.547859+0000","last_undegraded":"2024-05-11T12:23:11.547859+0000","last_fullsized":"2024-05-11T12:23:11.547859+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:02:46.357968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094221+0000","last_change":"2024-05-11T12:21:08.510504+0000","last_active":"2024-05-11T12:22:44.094221+0000","last_peered":"2024-05-11T12:22:44.094221+0000","last_clean":"2024-05-11T12:22:44.094221+0000","last_became_active":"2024-05-11T12:21:08.510074+0000","last_became_peered":"2024-05-11T12:21:08.510074+0000","last_unstale":"2024-05-11T12:22:44.094221+0000","last_undegraded":"2024-05-11T12:22:44.094221+0000","last_fullsized":"2024-05-11T12:22:44.094221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:31:49.465252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.555652+0000","last_change":"2024-05-11T12:22:43.753140+0000","last_active":"2024-05-11T12:23:11.555652+0000","last_peered":"2024-05-11T12:23:11.555652+0000","last_clean":"2024-05-11T12:23:11.555652+0000","last_became_active":"2024-05-11T12:22:43.752693+0000","last_became_peered":"2024-05-11T12:22:43.752693+0000","last_unstale":"2024-05-11T12:23:11.555652+0000","last_undegraded":"2024-05-11T12:23:11.555652+0000","last_fullsized":"2024-05-11T12:23:11.555652+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:46:13.152017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7],"acting":[11,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.647507+0000","last_change":"2024-05-11T12:21:08.531926+0000","last_active":"2024-05-11T12:22:43.647507+0000","last_peered":"2024-05-11T12:22:43.647507+0000","last_clean":"2024-05-11T12:22:43.647507+0000","last_became_active":"2024-05-11T12:21:08.531548+0000","last_became_peered":"2024-05-11T12:21:08.531548+0000","last_unstale":"2024-05-11T12:22:43.647507+0000","last_undegraded":"2024-05-11T12:22:43.647507+0000","last_fullsized":"2024-05-11T12:22:43.647507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:25:22.448630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548090+0000","last_change":"2024-05-11T12:21:11.528595+0000","last_active":"2024-05-11T12:23:11.548090+0000","last_peered":"2024-05-11T12:23:11.548090+0000","last_clean":"2024-05-11T12:23:11.548090+0000","last_became_active":"2024-05-11T12:21:11.527782+0000","last_became_peered":"2024-05-11T12:21:11.527782+0000","last_unstale":"2024-05-11T12:23:11.548090+0000","last_undegraded":"2024-05-11T12:23:11.548090+0000","last_fullsized":"2024-05-11T12:23:11.548090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:31:00.337497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649651+0000","last_change":"2024-05-11T12:21:08.508093+0000","last_active":"2024-05-11T12:22:43.649651+0000","last_peered":"2024-05-11T12:22:43.649651+0000","last_clean":"2024-05-11T12:22:43.649651+0000","last_became_active":"2024-05-11T12:21:08.507710+0000","last_became_peered":"2024-05-11T12:21:08.507710+0000","last_unstale":"2024-05-11T12:22:43.649651+0000","last_undegraded":"2024-05-11T12:22:43.649651+0000","last_fullsized":"2024-05-11T12:22:43.649651+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:55:24.273932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.557566+0000","last_change":"2024-05-11T12:21:11.526026+0000","last_active":"2024-05-11T12:23:11.557566+0000","last_peered":"2024-05-11T12:23:11.557566+0000","last_clean":"2024-05-11T12:23:11.557566+0000","last_became_active":"2024-05-11T12:21:11.525628+0000","last_became_peered":"2024-05-11T12:21:11.525628+0000","last_unstale":"2024-05-11T12:23:11.557566+0000","last_undegraded":"2024-05-11T12:23:11.557566+0000","last_fullsized":"2024-05-11T12:23:11.557566+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:51:12.789677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561475+0000","last_change":"2024-05-11T12:21:11.535808+0000","last_active":"2024-05-11T12:23:11.561475+0000","last_peered":"2024-05-11T12:23:11.561475+0000","last_clean":"2024-05-11T12:23:11.561475+0000","last_became_active":"2024-05-11T12:21:11.535204+0000","last_became_peered":"2024-05-11T12:21:11.535204+0000","last_unstale":"2024-05-11T12:23:11.561475+0000","last_undegraded":"2024-05-11T12:23:11.561475+0000","last_fullsized":"2024-05-11T12:23:11.561475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T22:36:01.208543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.844644+0000","last_change":"2024-05-11T12:21:08.531838+0000","last_active":"2024-05-11T12:22:43.844644+0000","last_peered":"2024-05-11T12:22:43.844644+0000","last_clean":"2024-05-11T12:22:43.844644+0000","last_became_active":"2024-05-11T12:21:08.531491+0000","last_became_peered":"2024-05-11T12:21:08.531491+0000","last_unstale":"2024-05-11T12:22:43.844644+0000","last_undegraded":"2024-05-11T12:22:43.844644+0000","last_fullsized":"2024-05-11T12:22:43.844644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:30:33.928985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.535726+0000","last_change":"2024-05-11T12:21:11.538563+0000","last_active":"2024-05-11T12:23:11.535726+0000","last_peered":"2024-05-11T12:23:11.535726+0000","last_clean":"2024-05-11T12:23:11.535726+0000","last_became_active":"2024-05-11T12:21:11.538200+0000","last_became_peered":"2024-05-11T12:21:11.538200+0000","last_unstale":"2024-05-11T12:23:11.535726+0000","last_undegraded":"2024-05-11T12:23:11.535726+0000","last_fullsized":"2024-05-11T12:23:11.535726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:35:18.879528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648409+0000","last_change":"2024-05-11T12:21:08.509972+0000","last_active":"2024-05-11T12:22:43.648409+0000","last_peered":"2024-05-11T12:22:43.648409+0000","last_clean":"2024-05-11T12:22:43.648409+0000","last_became_active":"2024-05-11T12:21:08.509531+0000","last_became_peered":"2024-05-11T12:21:08.509531+0000","last_unstale":"2024-05-11T12:22:43.648409+0000","last_undegraded":"2024-05-11T12:22:43.648409+0000","last_fullsized":"2024-05-11T12:22:43.648409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:47:13.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":105,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649321+0000","last_change":"2024-05-11T12:19:42.546860+0000","last_active":"2024-05-11T12:22:43.649321+0000","last_peered":"2024-05-11T12:22:43.649321+0000","last_clean":"2024-05-11T12:22:43.649321+0000","last_became_active":"2024-05-11T12:19:42.541449+0000","last_became_peered":"2024-05-11T12:19:42.541449+0000","last_unstale":"2024-05-11T12:22:43.649321+0000","last_undegraded":"2024-05-11T12:22:43.649321+0000","last_fullsized":"2024-05-11T12:22:43.649321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:16:12.381187+0000","last_clean_scrub_stamp":"2024-05-11T12:16:12.381187+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:25:45.938029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.559597+0000","last_change":"2024-05-11T12:21:11.536770+0000","last_active":"2024-05-11T12:23:11.559597+0000","last_peered":"2024-05-11T12:23:11.559597+0000","last_clean":"2024-05-11T12:23:11.559597+0000","last_became_active":"2024-05-11T12:21:11.536421+0000","last_became_peered":"2024-05-11T12:21:11.536421+0000","last_unstale":"2024-05-11T12:23:11.559597+0000","last_undegraded":"2024-05-11T12:23:11.559597+0000","last_fullsized":"2024-05-11T12:23:11.559597+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:13:42.217398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.649356+0000","last_change":"2024-05-11T12:21:42.675283+0000","last_active":"2024-05-11T12:22:43.649356+0000","last_peered":"2024-05-11T12:22:43.649356+0000","last_clean":"2024-05-11T12:22:43.649356+0000","last_became_active":"2024-05-11T12:21:42.674775+0000","last_became_peered":"2024-05-11T12:21:42.674775+0000","last_unstale":"2024-05-11T12:22:43.649356+0000","last_undegraded":"2024-05-11T12:22:43.649356+0000","last_fullsized":"2024-05-11T12:22:43.649356+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:13:30.143988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.094224+0000","last_change":"2024-05-11T12:21:08.533462+0000","last_active":"2024-05-11T12:22:44.094224+0000","last_peered":"2024-05-11T12:22:44.094224+0000","last_clean":"2024-05-11T12:22:44.094224+0000","last_became_active":"2024-05-11T12:21:08.533081+0000","last_became_peered":"2024-05-11T12:21:08.533081+0000","last_unstale":"2024-05-11T12:22:44.094224+0000","last_undegraded":"2024-05-11T12:22:44.094224+0000","last_fullsized":"2024-05-11T12:22:44.094224+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:01:48.286336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.549932+0000","last_change":"2024-05-11T12:21:11.539484+0000","last_active":"2024-05-11T12:23:11.549932+0000","last_peered":"2024-05-11T12:23:11.549932+0000","last_clean":"2024-05-11T12:23:11.549932+0000","last_became_active":"2024-05-11T12:21:11.536810+0000","last_became_peered":"2024-05-11T12:21:11.536810+0000","last_unstale":"2024-05-11T12:23:11.549932+0000","last_undegraded":"2024-05-11T12:23:11.549932+0000","last_fullsized":"2024-05-11T12:23:11.549932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:47:41.461766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'12","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:19.023392+0000","last_change":"2024-05-11T12:21:08.510519+0000","last_active":"2024-05-11T12:23:19.023392+0000","last_peered":"2024-05-11T12:23:19.023392+0000","last_clean":"2024-05-11T12:23:19.023392+0000","last_became_active":"2024-05-11T12:21:08.510180+0000","last_became_peered":"2024-05-11T12:21:08.510180+0000","last_unstale":"2024-05-11T12:23:19.023392+0000","last_undegraded":"2024-05-11T12:23:19.023392+0000","last_fullsized":"2024-05-11T12:23:19.023392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:22:24.537043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553050+0000","last_change":"2024-05-11T12:21:11.533726+0000","last_active":"2024-05-11T12:23:11.553050+0000","last_peered":"2024-05-11T12:23:11.553050+0000","last_clean":"2024-05-11T12:23:11.553050+0000","last_became_active":"2024-05-11T12:21:11.533393+0000","last_became_peered":"2024-05-11T12:21:11.533393+0000","last_unstale":"2024-05-11T12:23:11.553050+0000","last_undegraded":"2024-05-11T12:23:11.553050+0000","last_fullsized":"2024-05-11T12:23:11.553050+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:45:03.071666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"79'2","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.648390+0000","last_change":"2024-05-11T12:21:09.036622+0000","last_active":"2024-05-11T12:22:43.648390+0000","last_peered":"2024-05-11T12:22:43.648390+0000","last_clean":"2024-05-11T12:22:43.648390+0000","last_became_active":"2024-05-11T12:21:09.036209+0000","last_became_peered":"2024-05-11T12:21:09.036209+0000","last_unstale":"2024-05-11T12:22:43.648390+0000","last_undegraded":"2024-05-11T12:22:43.648390+0000","last_fullsized":"2024-05-11T12:22:43.648390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:29:58.947588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553776+0000","last_change":"2024-05-11T12:22:43.846111+0000","last_active":"2024-05-11T12:23:11.553776+0000","last_peered":"2024-05-11T12:23:11.553776+0000","last_clean":"2024-05-11T12:23:11.553776+0000","last_became_active":"2024-05-11T12:22:43.845712+0000","last_became_peered":"2024-05-11T12:22:43.845712+0000","last_unstale":"2024-05-11T12:23:11.553776+0000","last_undegraded":"2024-05-11T12:23:11.553776+0000","last_fullsized":"2024-05-11T12:23:11.553776+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:19:47.516917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.891608+0000","last_change":"2024-05-11T12:21:08.533435+0000","last_active":"2024-05-11T12:22:43.891608+0000","last_peered":"2024-05-11T12:22:43.891608+0000","last_clean":"2024-05-11T12:22:43.891608+0000","last_became_active":"2024-05-11T12:21:08.533100+0000","last_became_peered":"2024-05-11T12:21:08.533100+0000","last_unstale":"2024-05-11T12:22:43.891608+0000","last_undegraded":"2024-05-11T12:22:43.891608+0000","last_fullsized":"2024-05-11T12:22:43.891608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:49:17.619015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.529373+0000","last_change":"2024-05-11T12:21:11.526140+0000","last_active":"2024-05-11T12:23:11.529373+0000","last_peered":"2024-05-11T12:23:11.529373+0000","last_clean":"2024-05-11T12:23:11.529373+0000","last_became_active":"2024-05-11T12:21:11.525810+0000","last_became_peered":"2024-05-11T12:21:11.525810+0000","last_unstale":"2024-05-11T12:23:11.529373+0000","last_undegraded":"2024-05-11T12:23:11.529373+0000","last_fullsized":"2024-05-11T12:23:11.529373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:14:04.160379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.790116+0000","last_change":"2024-05-11T12:21:08.508031+0000","last_active":"2024-05-11T12:22:43.790116+0000","last_peered":"2024-05-11T12:22:43.790116+0000","last_clean":"2024-05-11T12:22:43.790116+0000","last_became_active":"2024-05-11T12:21:08.507703+0000","last_became_peered":"2024-05-11T12:21:08.507703+0000","last_unstale":"2024-05-11T12:22:43.790116+0000","last_undegraded":"2024-05-11T12:22:43.790116+0000","last_fullsized":"2024-05-11T12:22:43.790116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:33:05.939570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547615+0000","last_change":"2024-05-11T12:21:11.528293+0000","last_active":"2024-05-11T12:23:11.547615+0000","last_peered":"2024-05-11T12:23:11.547615+0000","last_clean":"2024-05-11T12:23:11.547615+0000","last_became_active":"2024-05-11T12:21:11.527862+0000","last_became_peered":"2024-05-11T12:21:11.527862+0000","last_unstale":"2024-05-11T12:23:11.547615+0000","last_undegraded":"2024-05-11T12:23:11.547615+0000","last_fullsized":"2024-05-11T12:23:11.547615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:10:07.335829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653717+0000","last_change":"2024-05-11T12:21:08.510138+0000","last_active":"2024-05-11T12:22:43.653717+0000","last_peered":"2024-05-11T12:22:43.653717+0000","last_clean":"2024-05-11T12:22:43.653717+0000","last_became_active":"2024-05-11T12:21:08.509566+0000","last_became_peered":"2024-05-11T12:21:08.509566+0000","last_unstale":"2024-05-11T12:22:43.653717+0000","last_undegraded":"2024-05-11T12:22:43.653717+0000","last_fullsized":"2024-05-11T12:22:43.653717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:24:58.174305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.542051+0000","last_change":"2024-05-11T12:22:43.750230+0000","last_active":"2024-05-11T12:23:11.542051+0000","last_peered":"2024-05-11T12:23:11.542051+0000","last_clean":"2024-05-11T12:23:11.542051+0000","last_became_active":"2024-05-11T12:22:43.749836+0000","last_became_peered":"2024-05-11T12:22:43.749836+0000","last_unstale":"2024-05-11T12:23:11.542051+0000","last_undegraded":"2024-05-11T12:23:11.542051+0000","last_fullsized":"2024-05-11T12:23:11.542051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:00:36.498862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"79'2","reported_seq":44,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:44.093949+0000","last_change":"2024-05-11T12:21:42.683706+0000","last_active":"2024-05-11T12:22:44.093949+0000","last_peered":"2024-05-11T12:22:44.093949+0000","last_clean":"2024-05-11T12:22:44.093949+0000","last_became_active":"2024-05-11T12:21:42.675791+0000","last_became_peered":"2024-05-11T12:21:42.675791+0000","last_unstale":"2024-05-11T12:22:44.093949+0000","last_undegraded":"2024-05-11T12:22:44.093949+0000","last_fullsized":"2024-05-11T12:22:44.093949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T16:13:26.737525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4],"acting":[8,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.560452+0000","last_change":"2024-05-11T12:21:11.544153+0000","last_active":"2024-05-11T12:23:11.560452+0000","last_peered":"2024-05-11T12:23:11.560452+0000","last_clean":"2024-05-11T12:23:11.560452+0000","last_became_active":"2024-05-11T12:21:11.543772+0000","last_became_peered":"2024-05-11T12:21:11.543772+0000","last_unstale":"2024-05-11T12:23:11.560452+0000","last_undegraded":"2024-05-11T12:23:11.560452+0000","last_fullsized":"2024-05-11T12:23:11.560452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:54:30.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.653862+0000","last_change":"2024-05-11T12:21:09.036409+0000","last_active":"2024-05-11T12:22:43.653862+0000","last_peered":"2024-05-11T12:22:43.653862+0000","last_clean":"2024-05-11T12:22:43.653862+0000","last_became_active":"2024-05-11T12:21:09.036009+0000","last_became_peered":"2024-05-11T12:21:09.036009+0000","last_unstale":"2024-05-11T12:22:43.653862+0000","last_undegraded":"2024-05-11T12:22:43.653862+0000","last_fullsized":"2024-05-11T12:22:43.653862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T18:55:28.736718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561529+0000","last_change":"2024-05-11T12:21:11.542685+0000","last_active":"2024-05-11T12:23:11.561529+0000","last_peered":"2024-05-11T12:23:11.561529+0000","last_clean":"2024-05-11T12:23:11.561529+0000","last_became_active":"2024-05-11T12:21:11.542130+0000","last_became_peered":"2024-05-11T12:21:11.542130+0000","last_unstale":"2024-05-11T12:23:11.561529+0000","last_undegraded":"2024-05-11T12:23:11.561529+0000","last_fullsized":"2024-05-11T12:23:11.561529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:44:22.578809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.751961+0000","last_change":"2024-05-11T12:21:08.510869+0000","last_active":"2024-05-11T12:22:43.751961+0000","last_peered":"2024-05-11T12:22:43.751961+0000","last_clean":"2024-05-11T12:22:43.751961+0000","last_became_active":"2024-05-11T12:21:08.510510+0000","last_became_peered":"2024-05-11T12:21:08.510510+0000","last_unstale":"2024-05-11T12:22:43.751961+0000","last_undegraded":"2024-05-11T12:22:43.751961+0000","last_fullsized":"2024-05-11T12:22:43.751961+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:44:50.002313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556787+0000","last_change":"2024-05-11T12:21:11.538622+0000","last_active":"2024-05-11T12:23:11.556787+0000","last_peered":"2024-05-11T12:23:11.556787+0000","last_clean":"2024-05-11T12:23:11.556787+0000","last_became_active":"2024-05-11T12:21:11.538229+0000","last_became_peered":"2024-05-11T12:21:11.538229+0000","last_unstale":"2024-05-11T12:23:11.556787+0000","last_undegraded":"2024-05-11T12:23:11.556787+0000","last_fullsized":"2024-05-11T12:23:11.556787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:01:15.912428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:22:43.845101+0000","last_change":"2024-05-11T12:21:42.677421+0000","last_active":"2024-05-11T12:22:43.845101+0000","last_peered":"2024-05-11T12:22:43.845101+0000","last_clean":"2024-05-11T12:22:43.845101+0000","last_became_active":"2024-05-11T12:21:42.672695+0000","last_became_peered":"2024-05-11T12:21:42.672695+0000","last_unstale":"2024-05-11T12:22:43.845101+0000","last_undegraded":"2024-05-11T12:22:43.845101+0000","last_fullsized":"2024-05-11T12:22:43.845101+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:07.492647+0000","last_clean_scrub_stamp":"2024-05-11T12:21:07.492647+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:30:27.606816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.556993+0000","last_change":"2024-05-11T12:21:11.536149+0000","last_active":"2024-05-11T12:23:11.556993+0000","last_peered":"2024-05-11T12:23:11.556993+0000","last_clean":"2024-05-11T12:23:11.556993+0000","last_became_active":"2024-05-11T12:21:11.535763+0000","last_became_peered":"2024-05-11T12:21:11.535763+0000","last_unstale":"2024-05-11T12:23:11.556993+0000","last_undegraded":"2024-05-11T12:23:11.556993+0000","last_fullsized":"2024-05-11T12:23:11.556993+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:46:15.246610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":34,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547785+0000","last_change":"2024-05-11T12:21:11.534318+0000","last_active":"2024-05-11T12:23:11.547785+0000","last_peered":"2024-05-11T12:23:11.547785+0000","last_clean":"2024-05-11T12:23:11.547785+0000","last_became_active":"2024-05-11T12:21:11.533949+0000","last_became_peered":"2024-05-11T12:21:11.533949+0000","last_unstale":"2024-05-11T12:23:11.547785+0000","last_undegraded":"2024-05-11T12:23:11.547785+0000","last_fullsized":"2024-05-11T12:23:11.547785+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:41:42.813949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561905+0000","last_change":"2024-05-11T12:21:11.529276+0000","last_active":"2024-05-11T12:23:11.561905+0000","last_peered":"2024-05-11T12:23:11.561905+0000","last_clean":"2024-05-11T12:23:11.561905+0000","last_became_active":"2024-05-11T12:21:11.528857+0000","last_became_peered":"2024-05-11T12:21:11.528857+0000","last_unstale":"2024-05-11T12:23:11.561905+0000","last_undegraded":"2024-05-11T12:23:11.561905+0000","last_fullsized":"2024-05-11T12:23:11.561905+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T21:07:54.270651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548623+0000","last_change":"2024-05-11T12:21:11.532342+0000","last_active":"2024-05-11T12:23:11.548623+0000","last_peered":"2024-05-11T12:23:11.548623+0000","last_clean":"2024-05-11T12:23:11.548623+0000","last_became_active":"2024-05-11T12:21:11.531949+0000","last_became_peered":"2024-05-11T12:21:11.531949+0000","last_unstale":"2024-05-11T12:23:11.548623+0000","last_undegraded":"2024-05-11T12:23:11.548623+0000","last_fullsized":"2024-05-11T12:23:11.548623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T13:23:00.010874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548470+0000","last_change":"2024-05-11T12:21:11.531547+0000","last_active":"2024-05-11T12:23:11.548470+0000","last_peered":"2024-05-11T12:23:11.548470+0000","last_clean":"2024-05-11T12:23:11.548470+0000","last_became_active":"2024-05-11T12:21:11.529702+0000","last_became_peered":"2024-05-11T12:21:11.529702+0000","last_unstale":"2024-05-11T12:23:11.548470+0000","last_undegraded":"2024-05-11T12:23:11.548470+0000","last_fullsized":"2024-05-11T12:23:11.548470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T15:11:13.857026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.554216+0000","last_change":"2024-05-11T12:21:11.531261+0000","last_active":"2024-05-11T12:23:11.554216+0000","last_peered":"2024-05-11T12:23:11.554216+0000","last_clean":"2024-05-11T12:23:11.554216+0000","last_became_active":"2024-05-11T12:21:11.530561+0000","last_became_peered":"2024-05-11T12:21:11.530561+0000","last_unstale":"2024-05-11T12:23:11.554216+0000","last_undegraded":"2024-05-11T12:23:11.554216+0000","last_fullsized":"2024-05-11T12:23:11.554216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T17:52:07.654417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.546445+0000","last_change":"2024-05-11T12:21:11.533255+0000","last_active":"2024-05-11T12:23:11.546445+0000","last_peered":"2024-05-11T12:23:11.546445+0000","last_clean":"2024-05-11T12:23:11.546445+0000","last_became_active":"2024-05-11T12:21:11.532909+0000","last_became_peered":"2024-05-11T12:21:11.532909+0000","last_unstale":"2024-05-11T12:23:11.546445+0000","last_undegraded":"2024-05-11T12:23:11.546445+0000","last_fullsized":"2024-05-11T12:23:11.546445+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:07:39.501482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553493+0000","last_change":"2024-05-11T12:21:11.528856+0000","last_active":"2024-05-11T12:23:11.553493+0000","last_peered":"2024-05-11T12:23:11.553493+0000","last_clean":"2024-05-11T12:23:11.553493+0000","last_became_active":"2024-05-11T12:21:11.528230+0000","last_became_peered":"2024-05-11T12:21:11.528230+0000","last_unstale":"2024-05-11T12:23:11.553493+0000","last_undegraded":"2024-05-11T12:23:11.553493+0000","last_fullsized":"2024-05-11T12:23:11.553493+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T19:25:40.429045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.553955+0000","last_change":"2024-05-11T12:21:11.534606+0000","last_active":"2024-05-11T12:23:11.553955+0000","last_peered":"2024-05-11T12:23:11.553955+0000","last_clean":"2024-05-11T12:23:11.553955+0000","last_became_active":"2024-05-11T12:21:11.534193+0000","last_became_peered":"2024-05-11T12:21:11.534193+0000","last_unstale":"2024-05-11T12:23:11.553955+0000","last_undegraded":"2024-05-11T12:23:11.553955+0000","last_fullsized":"2024-05-11T12:23:11.553955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:00:12.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.548067+0000","last_change":"2024-05-11T12:21:11.525959+0000","last_active":"2024-05-11T12:23:11.548067+0000","last_peered":"2024-05-11T12:23:11.548067+0000","last_clean":"2024-05-11T12:23:11.548067+0000","last_became_active":"2024-05-11T12:21:11.525129+0000","last_became_peered":"2024-05-11T12:21:11.525129+0000","last_unstale":"2024-05-11T12:23:11.548067+0000","last_undegraded":"2024-05-11T12:23:11.548067+0000","last_fullsized":"2024-05-11T12:23:11.548067+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:50:28.828336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.547068+0000","last_change":"2024-05-11T12:21:11.538591+0000","last_active":"2024-05-11T12:23:11.547068+0000","last_peered":"2024-05-11T12:23:11.547068+0000","last_clean":"2024-05-11T12:23:11.547068+0000","last_became_active":"2024-05-11T12:21:11.537855+0000","last_became_peered":"2024-05-11T12:21:11.537855+0000","last_unstale":"2024-05-11T12:23:11.547068+0000","last_undegraded":"2024-05-11T12:23:11.547068+0000","last_fullsized":"2024-05-11T12:23:11.547068+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T14:18:52.625763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.563270+0000","last_change":"2024-05-11T12:21:11.541185+0000","last_active":"2024-05-11T12:23:11.563270+0000","last_peered":"2024-05-11T12:23:11.563270+0000","last_clean":"2024-05-11T12:23:11.563270+0000","last_became_active":"2024-05-11T12:21:11.540799+0000","last_became_peered":"2024-05-11T12:21:11.540799+0000","last_unstale":"2024-05-11T12:23:11.563270+0000","last_undegraded":"2024-05-11T12:23:11.563270+0000","last_fullsized":"2024-05-11T12:23:11.563270+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T23:03:24.441232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.552798+0000","last_change":"2024-05-11T12:21:11.527811+0000","last_active":"2024-05-11T12:23:11.552798+0000","last_peered":"2024-05-11T12:23:11.552798+0000","last_clean":"2024-05-11T12:23:11.552798+0000","last_became_active":"2024-05-11T12:21:11.526603+0000","last_became_peered":"2024-05-11T12:21:11.526603+0000","last_unstale":"2024-05-11T12:23:11.552798+0000","last_undegraded":"2024-05-11T12:23:11.552798+0000","last_fullsized":"2024-05-11T12:23:11.552798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T20:46:43.833506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-05-11T12:23:11.561468+0000","last_change":"2024-05-11T12:21:11.530824+0000","last_active":"2024-05-11T12:23:11.561468+0000","last_peered":"2024-05-11T12:23:11.561468+0000","last_clean":"2024-05-11T12:23:11.561468+0000","last_became_active":"2024-05-11T12:21:11.530486+0000","last_became_peered":"2024-05-11T12:21:11.530486+0000","last_unstale":"2024-05-11T12:23:11.561468+0000","last_undegraded":"2024-05-11T12:23:11.561468+0000","last_fullsized":"2024-05-11T12:23:11.561468+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-11T12:21:10.501100+0000","last_clean_scrub_stamp":"2024-05-11T12:21:10.501100+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-12T12:30:47.054822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247249,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63105,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":331,"num_read_kb":229,"num_write":94,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":189315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678068,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":897024,"data_stored":445299,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":1098251,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037823,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445847,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":901120,"data_stored":445869,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430281,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36264,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708888,"statfs":{"total":95995035648,"available":95957901312,"internally_reserved":0,"allocated":950272,"data_stored":1036332,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445382,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":507572,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":445355,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":507572,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":909312,"data_stored":445603,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36212,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708940,"statfs":{"total":95995035648,"available":95957954560,"internally_reserved":0,"allocated":897024,"data_stored":445287,"data_compressed":11592,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":1035932,"data_compressed":47377,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34162,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35785,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-11T12:24:09.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:24:10.211 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:11.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:10 smithi047 ceph-mon[30375]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:10 smithi047 ceph-mon[30375]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:10 smithi047 ceph-mon[30375]: pgmap v387: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:24:11.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:10 smithi047 ceph-mon[30375]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:10 smithi064 ceph-mon[34537]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:10 smithi064 ceph-mon[34537]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:10 smithi064 ceph-mon[34537]: pgmap v387: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:24:11.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:10 smithi064 ceph-mon[34537]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:10 smithi050 ceph-mon[34072]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:10 smithi050 ceph-mon[34072]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:11.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:10 smithi050 ceph-mon[34072]: pgmap v387: 97 pgs: 97 active+clean; 638 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-05-11T12:24:11.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:10 smithi050 ceph-mon[34072]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-11T12:24:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 -- 172.21.15.47:0/531020962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc108530 msgr2=0x7faefc108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 --2- 172.21.15.47:0/531020962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc108530 0x7faefc108910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7faef8009990 tx=0x7faef8009ca0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 -- 172.21.15.47:0/531020962 shutdown_connections 2024-05-11T12:24:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 --2- 172.21.15.47:0/531020962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faefc102e10 0x7faefc10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 --2- 172.21.15.47:0/531020962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc108530 0x7faefc108910 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 --2- 172.21.15.47:0/531020962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faefc102450 0x7faefc1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 -- 172.21.15.47:0/531020962 >> 172.21.15.47:0/531020962 conn(0x7faefc079400 msgr2=0x7faefc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 -- 172.21.15.47:0/531020962 shutdown_connections 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.046+0000 7faf03f6a700 1 -- 172.21.15.47:0/531020962 wait complete. 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 Processor -- start 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 -- start start 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faefc102450 0x7faefc104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc102e10 0x7faefc1045d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faefc108530 0x7faefc104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faefc1126e0 con 0x7faefc102e10 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faefc112560 con 0x7faefc108530 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf03f6a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faefc112860 con 0x7faefc102450 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf01505700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc102e10 0x7faefc1045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:12.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf01505700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc102e10 0x7faefc1045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52454/0 (socket says 172.21.15.47:52454) 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.047+0000 7faf01505700 1 -- 172.21.15.47:0/561475301 learned_addr learned my addr 172.21.15.47:0/561475301 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf01d06700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faefc102450 0x7faefc104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf02507700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faefc108530 0x7faefc104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf01505700 1 -- 172.21.15.47:0/561475301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faefc102450 msgr2=0x7faefc104090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf01505700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faefc102450 0x7faefc104090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf01505700 1 -- 172.21.15.47:0/561475301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faefc108530 msgr2=0x7faefc104b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf01505700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faefc108530 0x7faefc104b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf01505700 1 -- 172.21.15.47:0/561475301 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc1a82a0 con 0x7faefc102e10 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf01505700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc102e10 0x7faefc1045d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faeec00b530 tx=0x7faeec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faef2ffd700 1 -- 172.21.15.47:0/561475301 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec015020 con 0x7faefc102e10 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faef2ffd700 1 -- 172.21.15.47:0/561475301 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeec011ce0 con 0x7faefc102e10 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc1a8590 con 0x7faefc102e10 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faef2ffd700 1 -- 172.21.15.47:0/561475301 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec010e40 con 0x7faefc102e10 2024-05-11T12:24:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.048+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc1a8b20 con 0x7faefc102e10 2024-05-11T12:24:12.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.049+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc06a240 con 0x7faefc102e10 2024-05-11T12:24:12.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.051+0000 7faef2ffd700 1 -- 172.21.15.47:0/561475301 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faeec018020 con 0x7faefc102e10 2024-05-11T12:24:12.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.051+0000 7faef2ffd700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faee80b2900 0x7faee80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:12.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.052+0000 7faf01d06700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faee80b2900 0x7faee80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:12.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.052+0000 7faef2ffd700 1 -- 172.21.15.47:0/561475301 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7faeec0e2050 con 0x7faefc102e10 2024-05-11T12:24:12.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.052+0000 7faf01d06700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faee80b2900 0x7faee80b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faef800b5c0 tx=0x7faef8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:12.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.055+0000 7faef2ffd700 1 -- 172.21.15.47:0/561475301 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faeec01f020 con 0x7faefc102e10 2024-05-11T12:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.318+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faefc1a8e30 con 0x7faefc102e10 2024-05-11T12:24:12.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.319+0000 7faef2ffd700 1 -- 172.21.15.47:0/561475301 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7faeec0a9ca0 con 0x7faefc102e10 2024-05-11T12:24:12.320 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:12.320 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:24:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faee80b2900 msgr2=0x7faee80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faee80b2900 0x7faee80b4dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faef800b5c0 tx=0x7faef8005960 comp rx=0 tx=0).stop 2024-05-11T12:24:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc102e10 msgr2=0x7faefc1045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc102e10 0x7faefc1045d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faeec00b530 tx=0x7faeec00b840 comp rx=0 tx=0).stop 2024-05-11T12:24:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 shutdown_connections 2024-05-11T12:24:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faee80b2900 0x7faee80b4dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faefc108530 0x7faefc104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faefc102450 0x7faefc104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 --2- 172.21.15.47:0/561475301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faefc102e10 0x7faefc1045d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 >> 172.21.15.47:0/561475301 conn(0x7faefc079400 msgr2=0x7faefc06b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 shutdown_connections 2024-05-11T12:24:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:12.321+0000 7faf03f6a700 1 -- 172.21.15.47:0/561475301 wait complete. 2024-05-11T12:24:12.323 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:24:13.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:24:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:12 smithi047 ceph-mon[30375]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-11T12:24:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:12 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/561475301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:13.288 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:12 smithi064 ceph-mon[34537]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-11T12:24:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:12 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/561475301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:13.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:12 smithi050 ceph-mon[34072]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-11T12:24:13.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:12 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/561475301' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 -- 172.21.15.47:0/10876538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 msgr2=0x7fab1806c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 --2- 172.21.15.47:0/10876538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 0x7fab1806c0e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fab14009990 tx=0x7fab14009ca0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 -- 172.21.15.47:0/10876538 shutdown_connections 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 --2- 172.21.15.47:0/10876538 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 0x7fab18112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 --2- 172.21.15.47:0/10876538 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 0x7fab1806c0e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 --2- 172.21.15.47:0/10876538 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1806c620 0x7fab1810fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 -- 172.21.15.47:0/10876538 >> 172.21.15.47:0/10876538 conn(0x7fab180794f0 msgr2=0x7fab18079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:15.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 -- 172.21.15.47:0/10876538 shutdown_connections 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.097+0000 7fab2024b700 1 -- 172.21.15.47:0/10876538 wait complete. 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 Processor -- start 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 -- start start 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 0x7fab1819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1806c620 0x7fab1819c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 0x7fab181a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fab18115970 con 0x7fab1810ffa0 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fab181157f0 con 0x7fab1806c620 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab2024b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fab18115af0 con 0x7fab1806bd00 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab1e7e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 0x7fab181a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:15.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab1e7e8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 0x7fab181a0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52466/0 (socket says 172.21.15.47:52466) 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab1e7e8700 1 -- 172.21.15.47:0/2808829894 learned_addr learned my addr 172.21.15.47:0/2808829894 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab1dfe7700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 0x7fab1819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.098+0000 7fab1d7e6700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1806c620 0x7fab1819c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab1e7e8700 1 -- 172.21.15.47:0/2808829894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 msgr2=0x7fab1819be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab1e7e8700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 0x7fab1819be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab1e7e8700 1 -- 172.21.15.47:0/2808829894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1806c620 msgr2=0x7fab1819c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab1e7e8700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1806c620 0x7fab1819c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab1e7e8700 1 -- 172.21.15.47:0/2808829894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab181a0e70 con 0x7fab1810ffa0 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab1e7e8700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 0x7fab181a0730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fab1000ef90 tx=0x7fab1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab0effd700 1 -- 172.21.15.47:0/2808829894 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab100092e0 con 0x7fab1810ffa0 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab181a1100 con 0x7fab1810ffa0 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab0effd700 1 -- 172.21.15.47:0/2808829894 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab10004bb0 con 0x7fab1810ffa0 2024-05-11T12:24:15.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab0effd700 1 -- 172.21.15.47:0/2808829894 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10004280 con 0x7fab1810ffa0 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.099+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab181a89e0 con 0x7fab1810ffa0 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.100+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1806a240 con 0x7fab1810ffa0 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.101+0000 7fab0effd700 1 -- 172.21.15.47:0/2808829894 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fab1001b030 con 0x7fab1810ffa0 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.102+0000 7fab0effd700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fab040b2880 0x7fab040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.103+0000 7fab1dfe7700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fab040b2880 0x7fab040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.103+0000 7fab0effd700 1 -- 172.21.15.47:0/2808829894 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fab100e2780 con 0x7fab1810ffa0 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.103+0000 7fab1dfe7700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fab040b2880 0x7fab040b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b2e0 tx=0x7fab14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:15.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.106+0000 7fab0effd700 1 -- 172.21.15.47:0/2808829894 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fab10022020 con 0x7fab1810ffa0 2024-05-11T12:24:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:14 smithi047 ceph-mon[30375]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-11T12:24:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.323+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab1802e570 con 0x7fab1810ffa0 2024-05-11T12:24:15.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.326+0000 7fab0effd700 1 -- 172.21.15.47:0/2808829894 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22917 (secure 0 0 0) 0x7fab10010040 con 0x7fab1810ffa0 2024-05-11T12:24:15.327 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:15.327 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:22:43.640272+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:24:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fab040b2880 msgr2=0x7fab040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fab040b2880 0x7fab040b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fab1400b2e0 tx=0x7fab14005960 comp rx=0 tx=0).stop 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 msgr2=0x7fab181a0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 0x7fab181a0730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fab1000ef90 tx=0x7fab1000c3a0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 shutdown_connections 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fab1810ffa0 0x7fab181a0730 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fab1806bd00 0x7fab1819be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fab040b2880 0x7fab040b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 --2- 172.21.15.47:0/2808829894 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fab1806c620 0x7fab1819c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.328+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 >> 172.21.15.47:0/2808829894 conn(0x7fab180794f0 msgr2=0x7fab181114b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.329+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 shutdown_connections 2024-05-11T12:24:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:15.329+0000 7fab2024b700 1 -- 172.21.15.47:0/2808829894 wait complete. 2024-05-11T12:24:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:14 smithi064 ceph-mon[34537]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-11T12:24:15.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:14 smithi050 ceph-mon[34072]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 767 B/s wr, 2 op/s 2024-05-11T12:24:15.972 DEBUG:tasks.fwd_scrub:Yielding 2024-05-11T12:24:15.972 INFO:teuthology.run_tasks:Running task workunit... 2024-05-11T12:24:15.989 INFO:tasks.workunit:Pulling workunits from ref 308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:24:15.989 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-11T12:24:15.989 INFO:tasks.workunit:timeout=3h 2024-05-11T12:24:15.989 INFO:tasks.workunit:cleanup=True 2024-05-11T12:24:15.990 DEBUG:teuthology.orchestra.run.smithi047:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-11T12:24:15.990 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-11T12:24:15.990 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:24:15.991 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 67h/103d Inode: 1099511628283 Links: 2 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-05-11 12:21:42.151922952 +0000 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-05-11 12:21:42.151922952 +0000 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-05-11 12:24:03.821072344 +0000 2024-05-11T12:24:16.009 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-05-11 12:21:42.151922952 +0000 2024-05-11T12:24:16.010 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-11T12:24:16.010 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-11T12:24:16.081 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 308506802b0bdea4df7050cfc1fcbdc04fea6a6e 2024-05-11T12:24:16.138 INFO:tasks.workunit.client.0.smithi047.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-11T12:24:16.254 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:16.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:15 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2808829894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:24:16.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:15 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2808829894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:24:16.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:15 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2808829894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:24:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:16 smithi047 ceph-mon[30375]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-11T12:24:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:16 smithi064 ceph-mon[34537]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-11T12:24:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:16 smithi050 ceph-mon[34072]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 767 B/s wr, 3 op/s 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 -- 172.21.15.47:0/763711987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 msgr2=0x7fcea4108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 --2- 172.21.15.47:0/763711987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4108a20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fce9400b0d0 tx=0x7fce9400b3e0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 -- 172.21.15.47:0/763711987 shutdown_connections 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 --2- 172.21.15.47:0/763711987 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcea4103160 0x7fcea410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 --2- 172.21.15.47:0/763711987 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4108a20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 --2- 172.21.15.47:0/763711987 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea41027a0 0x7fcea4102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 -- 172.21.15.47:0/763711987 >> 172.21.15.47:0/763711987 conn(0x7fcea40782c0 msgr2=0x7fcea40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:17.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.977+0000 7fceac7d3700 1 -- 172.21.15.47:0/763711987 shutdown_connections 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 -- 172.21.15.47:0/763711987 wait complete. 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 Processor -- start 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 -- start start 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcea41027a0 0x7fcea4104600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea4103160 0x7fcea4104b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4105080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41126a0 con 0x7fcea4108640 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4112520 con 0x7fcea41027a0 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.978+0000 7fceac7d3700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcea4112820 con 0x7fcea4103160 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4105080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4105080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46736/0 (socket says 172.21.15.47:46736) 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 -- 172.21.15.47:0/1481088312 learned_addr learned my addr 172.21.15.47:0/1481088312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea4103160 0x7fcea4104b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaa56f700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcea41027a0 0x7fcea4104600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 -- 172.21.15.47:0/1481088312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea4103160 msgr2=0x7fcea4104b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea4103160 0x7fcea4104b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 -- 172.21.15.47:0/1481088312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcea41027a0 msgr2=0x7fcea4104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcea41027a0 0x7fcea4104600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 -- 172.21.15.47:0/1481088312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea41a8140 con 0x7fcea4108640 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaa56f700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcea41027a0 0x7fcea4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceaad70700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4105080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00c970 tx=0x7fce9c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fcea37fe700 1 -- 172.21.15.47:0/1481088312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c00ce60 con 0x7fcea4108640 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fcea37fe700 1 -- 172.21.15.47:0/1481088312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce9c004830 con 0x7fcea4108640 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea41a8430 con 0x7fcea4108640 2024-05-11T12:24:17.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.979+0000 7fcea37fe700 1 -- 172.21.15.47:0/1481088312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0054c0 con 0x7fcea4108640 2024-05-11T12:24:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.980+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea41a8970 con 0x7fcea4108640 2024-05-11T12:24:17.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.982+0000 7fcea37fe700 1 -- 172.21.15.47:0/1481088312 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce9c0049e0 con 0x7fcea4108640 2024-05-11T12:24:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.983+0000 7fcea37fe700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce8c0b2900 0x7fce8c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.983+0000 7fceaa56f700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce8c0b2900 0x7fce8c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.983+0000 7fcea37fe700 1 -- 172.21.15.47:0/1481088312 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fce9c0e1d10 con 0x7fcea4108640 2024-05-11T12:24:17.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.983+0000 7fceb7069b80 1 Processor -- start 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceaa56f700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce8c0b2900 0x7fce8c0b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce94009fd0 tx=0x7fce9400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceb7069b80 1 -- start start 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b13f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0e120 0x55e6f4b0e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceb7069b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e6f4a608e0 con 0x55e6f4b0e120 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceb7069b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e6f49ff840 con 0x55e6f4b13b50 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceb7069b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e6f4a5c8a0 con 0x55e6f4b0eae0 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0e120 0x55e6f4b0e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0e120 0x55e6f4b0e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46748/0 (socket says 172.21.15.47:46748) 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 -- 172.21.15.47:0/1402865113 learned_addr learned my addr 172.21.15.47:0/1402865113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:17.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceaa56f700 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b13f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fceaad70700 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 -- 172.21.15.47:0/1402865113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 msgr2=0x55e6f4b1a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 -- 172.21.15.47:0/1402865113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 msgr2=0x55e6f4b13f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b13f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.984+0000 7fcea9d6e700 1 -- 172.21.15.47:0/1402865113 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f49ad950 con 0x55e6f4b0e120 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.985+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0e120 0x55e6f4b0e5a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fce9800bcf0 tx=0x7fce980103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48813fcb9902e44b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.985+0000 7fcea0ff9700 1 -- 172.21.15.47:0/1402865113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98010cb0 con 0x55e6f4b0e120 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.985+0000 7fcea0ff9700 1 -- 172.21.15.47:0/1402865113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce98010e50 con 0x55e6f4b0e120 2024-05-11T12:24:17.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.985+0000 7fcea0ff9700 1 -- 172.21.15.47:0/1402865113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98005280 con 0x55e6f4b0e120 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.985+0000 7fceb7069b80 1 -- 172.21.15.47:0/1402865113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0e120 msgr2=0x55e6f4b0e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.985+0000 7fceb7069b80 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0e120 0x55e6f4b0e5a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fce9800bcf0 tx=0x7fce980103d0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- 172.21.15.47:0/1402865113 shutdown_connections 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b13f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 --2- 172.21.15.47:0/1402865113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0e120 0x55e6f4b0e5a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- 172.21.15.47:0/1402865113 >> 172.21.15.47:0/1402865113 conn(0x55e6f4a3aee0 msgr2=0x55e6f4b08a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- 172.21.15.47:0/1402865113 shutdown_connections 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- 172.21.15.47:0/1402865113 wait complete. 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- start start 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 Processor -- start 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b1a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b17950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b13b50 0x55e6f4b17e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e6f4b1af10 con 0x55e6f4b13b50 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e6f49c6cb0 con 0x55e6f4b0e120 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e6f4b1b090 con 0x55e6f4b0eae0 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.986+0000 7fceb7069b80 10 client.0 _ll_register_callbacks cb 0x55e6f49f89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b13b50 0x55e6f4b17e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b13b50 0x55e6f4b17e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46750/0 (socket says 172.21.15.47:46750) 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 -- 172.21.15.47:0/4185298270 learned_addr learned my addr 172.21.15.47:0/4185298270 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b17950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaa56f700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b1a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 -- 172.21.15.47:0/4185298270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 msgr2=0x55e6f4b17950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b17950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 -- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 msgr2=0x55e6f4b1a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b1a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 -- 172.21.15.47:0/4185298270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f49fc710 con 0x55e6f4b13b50 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b17950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceaad70700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b13b50 0x55e6f4b17e90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0df740 tx=0x7fce9c0a9260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36a19c607de7a0c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fce89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fce89ffb700 1 -- 172.21.15.47:0/4185298270 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0c3060 con 0x55e6f4b13b50 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fce89ffb700 1 -- 172.21.15.47:0/4185298270 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fce9c0b1c30 con 0x55e6f4b13b50 2024-05-11T12:24:17.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce98010e50 con 0x55e6f4b13b50 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fce89ffb700 1 -- 172.21.15.47:0/4185298270 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce9c01d070 con 0x55e6f4b13b50 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceb7069b80 10 client.15123 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.987+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e6f4b25800 con 0x55e6f4b13b50 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fce89ffb700 1 -- 172.21.15.47:0/4185298270 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce98010e50 con 0x55e6f4b13b50 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fce89ffb700 1 -- 172.21.15.47:0/4185298270 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fce9c0ad290 con 0x55e6f4b13b50 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fceb7069b80 10 client.15123 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fceb7069b80 10 client.15123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fceb7069b80 20 client.15123 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b27d90 0x55e6f4b2a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fceb7069b80 4 client.15123 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f4a6e2c0 con 0x55e6f4b27d90 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.988+0000 7fceaa56f700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b27d90 0x55e6f4b2a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceaa56f700 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b27d90 0x55e6f4b2a180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de36d1e0c9424277 server_cookie=bc797e2826afffba in_seq=0 out_seq=0 2024-05-11T12:24:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fce89ffb700 10 client.15123 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fce89ffb700 1 -- 172.21.15.47:0/4185298270 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6f4a6e2c0 con 0x55e6f4b27d90 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fce89ffb700 10 client.15123 handle_command_reply: tid=0 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 client.15123 shutdown 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b13b50 msgr2=0x55e6f4b17e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b13b50 0x55e6f4b17e90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0df740 tx=0x7fce9c0a9260 comp rx=0 tx=0).stop 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 shutdown_connections 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b27d90 0x55e6f4b2a180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b13b50 0x55e6f4b17e90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b1a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.989+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4185298270 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b17950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.990+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 >> 172.21.15.47:0/4185298270 conn(0x55e6f4a3aee0 msgr2=0x55e6f4b034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.990+0000 7fce89ffb700 0 client.15123 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.990+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 shutdown_connections 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.990+0000 7fceb7069b80 1 -- 172.21.15.47:0/4185298270 wait complete. 2024-05-11T12:24:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:17.990+0000 7fceb7069b80 20 client.15123 trim_cache size 0 max 16384 2024-05-11T12:24:18.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 Processor -- start 2024-05-11T12:24:18.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 -- start start 2024-05-11T12:24:18.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b23fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b13b50 0x55e6f4b13320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ae130 con 0x55e6f4b0eae0 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41b09b0 con 0x55e6f4b0e120 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.011+0000 7fceb7069b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41ae7a0 con 0x55e6f4b13b50 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b23fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b23fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46766/0 (socket says 172.21.15.47:46766) 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 -- 172.21.15.47:0/2281320325 learned_addr learned my addr 172.21.15.47:0/2281320325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 -- 172.21.15.47:0/2281320325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b13b50 msgr2=0x55e6f4b13320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fceaa56f700 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b13b50 0x55e6f4b13320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 -- 172.21.15.47:0/2281320325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 msgr2=0x55e6f4b168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 -- 172.21.15.47:0/2281320325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f4b05230 con 0x55e6f4b0eae0 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fceaa56f700 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b23fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fce9800bc10 tx=0x7fce98005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdffff75ebbd10c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fce8bfff700 1 -- 172.21.15.47:0/2281320325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98010cb0 con 0x55e6f4b0eae0 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fce8bfff700 1 -- 172.21.15.47:0/2281320325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce98010e50 con 0x55e6f4b0eae0 2024-05-11T12:24:18.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.012+0000 7fce8bfff700 1 -- 172.21.15.47:0/2281320325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce980053f0 con 0x55e6f4b0eae0 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 -- 172.21.15.47:0/2281320325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 msgr2=0x55e6f4b23fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b23fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fce9800bc10 tx=0x7fce98005d20 comp rx=0 tx=0).stop 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 -- 172.21.15.47:0/2281320325 shutdown_connections 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b13b50 0x55e6f4b13320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b0e120 0x55e6f4b168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 --2- 172.21.15.47:0/2281320325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b23fa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 -- 172.21.15.47:0/2281320325 >> 172.21.15.47:0/2281320325 conn(0x55e6f4a3aee0 msgr2=0x55e6f4b0d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 -- 172.21.15.47:0/2281320325 shutdown_connections 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 -- 172.21.15.47:0/2281320325 wait complete. 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 -- start start 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.013+0000 7fceb7069b80 1 Processor -- start 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0e120 0x55e6f4b20510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:18.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceb7069b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b1b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceb7069b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41aee70 con 0x55e6f4b0eae0 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceb7069b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41b02e0 con 0x55e6f4b13b50 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceb7069b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcea41af540 con 0x55e6f4b0e120 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceb7069b80 10 client.0 _ll_register_callbacks cb 0x55e6f49f89a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46772/0 (socket says 172.21.15.47:46772) 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 -- 172.21.15.47:0/4058839387 learned_addr learned my addr 172.21.15.47:0/4058839387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceaa56f700 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0e120 0x55e6f4b20510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fceaad70700 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b1b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 -- 172.21.15.47:0/4058839387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0e120 msgr2=0x55e6f4b20510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0e120 0x55e6f4b20510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 -- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 msgr2=0x55e6f4b1b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:18.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b1b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 -- 172.21.15.47:0/4058839387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f4b1bd60 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fcea9d6e700 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1b2e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fce98005220 tx=0x7fce98005250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7053e2ef658bf19 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.014+0000 7fce89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fce89ffb700 1 -- 172.21.15.47:0/4058839387 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce98034060 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fce89ffb700 1 -- 172.21.15.47:0/4058839387 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fce98005630 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce98010e50 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fce89ffb700 1 -- 172.21.15.47:0/4058839387 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce98010c80 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 10 client.15129 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e6f4b931f0 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fce89ffb700 1 -- 172.21.15.47:0/4058839387 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce98010e50 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fce89ffb700 1 -- 172.21.15.47:0/4058839387 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fce98024070 con 0x55e6f4b0eae0 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 10 client.15129 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 10 client.15129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 20 client.15129 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:18.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b93660 0x55e6f4b95a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 4 client.15129 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6f4a68360 con 0x55e6f4b93660 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.015+0000 7fceaa56f700 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b93660 0x55e6f4b95a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.016+0000 7fceaa56f700 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b93660 0x55e6f4b95a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9dbbfca6cadc1b server_cookie=53faec8dad962624 in_seq=0 out_seq=0 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.016+0000 7fce89ffb700 10 client.15129 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fce89ffb700 1 -- 172.21.15.47:0/4058839387 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6f4a68360 con 0x55e6f4b93660 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fce89ffb700 10 client.15129 handle_command_reply: tid=0 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 client.15129 shutdown 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 msgr2=0x55e6f4b1b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1b2e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fce98005220 tx=0x7fce98005250 comp rx=0 tx=0).stop 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 shutdown_connections 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e6f4b93660 0x55e6f4b95a80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e6f4b13b50 0x55e6f4b1b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fce89ffb700 0 client.15129 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:18.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e6f4b0e120 0x55e6f4b20510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.017+0000 7fceb7069b80 1 --2- 172.21.15.47:0/4058839387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e6f4b0eae0 0x55e6f4b1b2e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.018+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 >> 172.21.15.47:0/4058839387 conn(0x55e6f4a3aee0 msgr2=0x55e6f4b0d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:18.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.018+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 shutdown_connections 2024-05-11T12:24:18.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.018+0000 7fceb7069b80 1 -- 172.21.15.47:0/4058839387 wait complete. 2024-05-11T12:24:18.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.018+0000 7fceb7069b80 20 client.15129 trim_cache size 0 max 16384 2024-05-11T12:24:18.024 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:24:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:24:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "59ddd64d-124a-4550-8df3-1d0c5cf345bd", 2024-05-11T12:24:18.025 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:24:18.025 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:24:18.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce8c0b2900 msgr2=0x7fce8c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce8c0b2900 0x7fce8c0b4dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fce94009fd0 tx=0x7fce9400b040 comp rx=0 tx=0).stop 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 msgr2=0x7fcea4105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4105080 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fce9c00c970 tx=0x7fce9c00cc80 comp rx=0 tx=0).stop 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 shutdown_connections 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcea4108640 0x7fcea4105080 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce8c0b2900 0x7fce8c0b4dc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcea41027a0 0x7fcea4104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 --2- 172.21.15.47:0/1481088312 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcea4103160 0x7fcea4104b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 >> 172.21.15.47:0/1481088312 conn(0x7fcea40782c0 msgr2=0x7fcea4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 shutdown_connections 2024-05-11T12:24:18.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:18.019+0000 7fceac7d3700 1 -- 172.21.15.47:0/1481088312 wait complete. 2024-05-11T12:24:18.723 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:24:19.028 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:19.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:18 smithi047 ceph-mon[30375]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-05-11T12:24:19.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:18 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:24:19.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:18 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:19.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:18 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:24:19.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:18 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:19.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:18 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:24:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:18 smithi064 ceph-mon[34537]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-05-11T12:24:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:18 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:24:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:18 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:18 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:24:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:18 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:18 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:24:19.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:18 smithi050 ceph-mon[34072]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 597 B/s wr, 1 op/s 2024-05-11T12:24:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:18 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:24:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:18 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:18 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:24:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:18 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:18 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:24:20.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 -- 172.21.15.47:0/928198503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b006bbf0 msgr2=0x7f96b01103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 --2- 172.21.15.47:0/928198503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b006bbf0 0x7f96b01103e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b0a0 tx=0x7f96a000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 -- 172.21.15.47:0/928198503 shutdown_connections 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 --2- 172.21.15.47:0/928198503 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b006bbf0 0x7f96b01103e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 --2- 172.21.15.47:0/928198503 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b01095c0 0x7f96b01099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 --2- 172.21.15.47:0/928198503 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b006b230 0x7f96b006b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 -- 172.21.15.47:0/928198503 >> 172.21.15.47:0/928198503 conn(0x7f96b0079400 msgr2=0x7f96b0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 -- 172.21.15.47:0/928198503 shutdown_connections 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.777+0000 7f96b595c700 1 -- 172.21.15.47:0/928198503 wait complete. 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 Processor -- start 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 -- start start 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b006b230 0x7f96b019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b006bbf0 0x7f96b019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b01095c0 0x7f96b0199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01137d0 con 0x7f96b01095c0 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0113650 con 0x7f96b006bbf0 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96b595c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0113950 con 0x7f96b006b230 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b01095c0 0x7f96b0199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b01095c0 0x7f96b0199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46790/0 (socket says 172.21.15.47:46790) 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.778+0000 7f96af7fe700 1 -- 172.21.15.47:0/2977766511 learned_addr learned my addr 172.21.15.47:0/2977766511 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b006bbf0 0x7f96b019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96aeffd700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b006b230 0x7f96b019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96af7fe700 1 -- 172.21.15.47:0/2977766511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b006b230 msgr2=0x7f96b019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96af7fe700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b006b230 0x7f96b019f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96af7fe700 1 -- 172.21.15.47:0/2977766511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b006bbf0 msgr2=0x7f96b019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96af7fe700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b006bbf0 0x7f96b019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96af7fe700 1 -- 172.21.15.47:0/2977766511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b019a250 con 0x7f96b01095c0 2024-05-11T12:24:20.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b006bbf0 0x7f96b019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96af7fe700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b01095c0 0x7f96b0199af0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b070 tx=0x7f96a000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96b495a700 1 -- 172.21.15.47:0/2977766511 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0021070 con 0x7f96b01095c0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96b495a700 1 -- 172.21.15.47:0/2977766511 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a0007cf0 con 0x7f96b01095c0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b019a4e0 con 0x7f96b01095c0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96b495a700 1 -- 172.21.15.47:0/2977766511 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0005440 con 0x7f96b01095c0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.779+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b01a67b0 con 0x7f96b01095c0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.781+0000 7f96b495a700 1 -- 172.21.15.47:0/2977766511 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f96a0024070 con 0x7f96b01095c0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.782+0000 7f96b495a700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f969c0b2900 0x7f969c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.782+0000 7f96aeffd700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f969c0b2900 0x7f969c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.782+0000 7f96b495a700 1 -- 172.21.15.47:0/2977766511 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f96a000f020 con 0x7f96b01095c0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 Processor -- start 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 -- start start 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96aeffd700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f969c0b2900 0x7f969c0b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005950 tx=0x7f96a8005d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:20.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36bc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555bf360a860 con 0x555bf36bc4b0 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555bf35a97c0 con 0x555bf36c1ee0 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bf3606820 con 0x555bf36bce70 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96ae7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36bc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96ae7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36bc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46802/0 (socket says 172.21.15.47:46802) 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1376808162 learned_addr learned my addr 172.21.15.47:0/1376808162 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96af7fe700 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.783+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1376808162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 msgr2=0x555bf36c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1376808162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 msgr2=0x555bf36c22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1376808162 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf35578d0 con 0x555bf36bc4b0 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36bc930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f969800aa70 tx=0x7f969800ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9240212397812052 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96a5ffb700 1 -- 172.21.15.47:0/1376808162 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698015070 con 0x555bf36bc4b0 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96a5ffb700 1 -- 172.21.15.47:0/1376808162 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96980113f0 con 0x555bf36bc4b0 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.784+0000 7f96a5ffb700 1 -- 172.21.15.47:0/1376808162 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980185c0 con 0x555bf36bc4b0 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1376808162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 msgr2=0x555bf36bc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36bc930 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f969800aa70 tx=0x7f969800ad80 comp rx=0 tx=0).stop 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1376808162 shutdown_connections 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1376808162 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36bc930 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1376808162 >> 172.21.15.47:0/1376808162 conn(0x555bf35e4e60 msgr2=0x555bf36b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1376808162 shutdown_connections 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1376808162 wait complete. 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 -- start start 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 Processor -- start 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.785+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bc4b0 0x555bf36c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bce70 0x555bf36c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555bf36c92a0 con 0x555bf36bce70 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555bf3570c30 con 0x555bf36c1ee0 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555bf36c9420 con 0x555bf36bc4b0 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96c01f2b80 10 client.0 _ll_register_callbacks cb 0x555bf35a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:20.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bce70 0x555bf36c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bce70 0x555bf36c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46816/0 (socket says 172.21.15.47:46816) 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1105270484 learned_addr learned my addr 172.21.15.47:0/1105270484 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96af7fe700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bc4b0 0x555bf36c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1105270484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bc4b0 msgr2=0x555bf36c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bc4b0 0x555bf36c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 msgr2=0x555bf36c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 -- 172.21.15.47:0/1105270484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf35a6690 con 0x555bf36bce70 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bc4b0 0x555bf36c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bce70 0x555bf36c5ca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9698006de0 tx=0x7f9698006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8944f8af79340994 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f968effd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:20.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f968effd700 1 -- 172.21.15.47:0/1105270484 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698015040 con 0x555bf36bce70 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.786+0000 7f968effd700 1 -- 172.21.15.47:0/1105270484 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f969800cc40 con 0x555bf36bce70 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96980113f0 con 0x555bf36bce70 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f968effd700 1 -- 172.21.15.47:0/1105270484 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f969800ce40 con 0x555bf36bce70 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 10 client.15141 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555bf36d3a70 con 0x555bf36bce70 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f968effd700 1 -- 172.21.15.47:0/1105270484 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96980113f0 con 0x555bf36bce70 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f968effd700 1 -- 172.21.15.47:0/1105270484 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f96980255b0 con 0x555bf36bce70 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 10 client.15141 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 10 client.15141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 20 client.15141 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf36d5f40 0x555bf36d8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 4 client.15141 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:24:20.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.787+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bf3618240 con 0x555bf36d5f40 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.788+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf36d5f40 0x555bf36d8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.788+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf36d5f40 0x555bf36d8330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8a7786e130be465 server_cookie=f0ac232e27c67e4d in_seq=0 out_seq=0 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.788+0000 7f968effd700 10 client.15141 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f968effd700 1 -- 172.21.15.47:0/1105270484 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555bf3618240 con 0x555bf36d5f40 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f968effd700 10 client.15141 handle_command_reply: tid=0 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 client.15141 shutdown 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bce70 msgr2=0x555bf36c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bce70 0x555bf36c5ca0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9698006de0 tx=0x7f9698006e10 comp rx=0 tx=0).stop 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 shutdown_connections 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf36d5f40 0x555bf36d8330 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bc4b0 0x555bf36c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f968effd700 0 client.15141 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:20.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1105270484 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bce70 0x555bf36c5ca0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 >> 172.21.15.47:0/1105270484 conn(0x555bf35e4e60 msgr2=0x555bf36ad460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:20.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 shutdown_connections 2024-05-11T12:24:20.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1105270484 wait complete. 2024-05-11T12:24:20.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.789+0000 7f96c01f2b80 20 client.15141 trim_cache size 0 max 16384 2024-05-11T12:24:20.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 Processor -- start 2024-05-11T12:24:20.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 -- start start 2024-05-11T12:24:20.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36bce70 0x555bf36c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36c1ee0 0x555bf36c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01a9c60 con 0x555bf36bc4b0 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01ac4e0 con 0x555bf36bce70 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01aa2d0 con 0x555bf36c1ee0 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46826/0 (socket says 172.21.15.47:46826) 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 -- 172.21.15.47:0/1282060221 learned_addr learned my addr 172.21.15.47:0/1282060221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36bce70 0x555bf36c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96af7fe700 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36c1ee0 0x555bf36c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 -- 172.21.15.47:0/1282060221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36c1ee0 msgr2=0x555bf36c12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36c1ee0 0x555bf36c12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 -- 172.21.15.47:0/1282060221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36bce70 msgr2=0x555bf36c0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36bce70 0x555bf36c0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96aeffd700 1 -- 172.21.15.47:0/1282060221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf36af1b0 con 0x555bf36bc4b0 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.810+0000 7f96af7fe700 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36c1ee0 0x555bf36c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96aeffd700 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36d22f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f96a800da80 tx=0x7f96a8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ba72105e43cd7e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96a57fa700 1 -- 172.21.15.47:0/1282060221 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a801d840 con 0x555bf36bc4b0 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96a57fa700 1 -- 172.21.15.47:0/1282060221 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a8004830 con 0x555bf36bc4b0 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96a57fa700 1 -- 172.21.15.47:0/1282060221 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80053e0 con 0x555bf36bc4b0 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1282060221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 msgr2=0x555bf36d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36d22f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f96a800da80 tx=0x7f96a8013000 comp rx=0 tx=0).stop 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1282060221 shutdown_connections 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36c1ee0 0x555bf36c12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36d22f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/1282060221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36bce70 0x555bf36c0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1282060221 >> 172.21.15.47:0/1282060221 conn(0x555bf35e4e60 msgr2=0x555bf36c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.811+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1282060221 shutdown_connections 2024-05-11T12:24:20.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 -- 172.21.15.47:0/1282060221 wait complete. 2024-05-11T12:24:20.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 -- start start 2024-05-11T12:24:20.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 Processor -- start 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01aa9a0 con 0x555bf36bc4b0 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01abe10 con 0x555bf36c1ee0 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f96b01ab070 con 0x555bf36bce70 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96c01f2b80 10 client.0 _ll_register_callbacks cb 0x555bf35a2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96aeffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46838/0 (socket says 172.21.15.47:46838) 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.812+0000 7f96aeffd700 1 -- 172.21.15.47:0/3606621095 learned_addr learned my addr 172.21.15.47:0/3606621095 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96af7fe700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96aeffd700 1 -- 172.21.15.47:0/3606621095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 msgr2=0x555bf36dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96aeffd700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96aeffd700 1 -- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 msgr2=0x555bf36dde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96aeffd700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36dde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96aeffd700 1 -- 172.21.15.47:0/3606621095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf36ccce0 con 0x555bf36bc4b0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96aeffd700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36dd3b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f96a801df70 tx=0x7f96a8004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6c9a98ca846dd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f968effd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f968effd700 1 -- 172.21.15.47:0/3606621095 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8034020 con 0x555bf36bc4b0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f968effd700 1 -- 172.21.15.47:0/3606621095 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f96a801da50 con 0x555bf36bc4b0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a8004830 con 0x555bf36bc4b0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f968effd700 1 -- 172.21.15.47:0/3606621095 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f96a803f050 con 0x555bf36bc4b0 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96c01f2b80 10 client.15147 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:20.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.813+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555bf3740a00 con 0x555bf36bc4b0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f968effd700 1 -- 172.21.15.47:0/3606621095 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96a8004830 con 0x555bf36bc4b0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f968effd700 1 -- 172.21.15.47:0/3606621095 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f96a8026560 con 0x555bf36bc4b0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96c01f2b80 10 client.15147 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96c01f2b80 10 client.15147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96c01f2b80 20 client.15147 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf37410b0 0x555bf37434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96c01f2b80 4 client.15147 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bf36122e0 con 0x555bf37410b0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf37410b0 0x555bf37434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f96ae7fc700 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf37410b0 0x555bf37434d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffeb2a135085fcc8 server_cookie=593b662630729cc2 in_seq=0 out_seq=0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.814+0000 7f968effd700 10 client.15147 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f968effd700 1 -- 172.21.15.47:0/3606621095 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555bf36122e0 con 0x555bf37410b0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f968effd700 10 client.15147 handle_command_reply: tid=0 2024-05-11T12:24:20.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 client.15147 shutdown 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 msgr2=0x555bf36dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36dd3b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f96a801df70 tx=0x7f96a8004cc0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 shutdown_connections 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555bf37410b0 0x555bf37434d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555bf36c1ee0 0x555bf36dde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555bf36bc4b0 0x555bf36dd3b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 --2- 172.21.15.47:0/3606621095 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555bf36bce70 0x555bf36dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 >> 172.21.15.47:0/3606621095 conn(0x555bf35e4e60 msgr2=0x555bf36c0db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f968effd700 0 client.15147 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 shutdown_connections 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 1 -- 172.21.15.47:0/3606621095 wait complete. 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.815+0000 7f96c01f2b80 20 client.15147 trim_cache size 0 max 16384 2024-05-11T12:24:20.821 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:24:20.822 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:24:20.822 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:24:20.822 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f969c0b2900 msgr2=0x7f969c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f969c0b2900 0x7f969c0b4dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f96a8005950 tx=0x7f96a8005d10 comp rx=0 tx=0).stop 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b01095c0 msgr2=0x7f96b0199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b01095c0 0x7f96b0199af0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b070 tx=0x7f96a000cf00 comp rx=0 tx=0).stop 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 shutdown_connections 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f96b01095c0 0x7f96b0199af0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f96b006b230 0x7f96b019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f969c0b2900 0x7f969c0b4dc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 --2- 172.21.15.47:0/2977766511 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f96b006bbf0 0x7f96b019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 >> 172.21.15.47:0/2977766511 conn(0x7f96b0079400 msgr2=0x7f96b01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 shutdown_connections 2024-05-11T12:24:20.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:20.816+0000 7f96b595c700 1 -- 172.21.15.47:0/2977766511 wait complete. 2024-05-11T12:24:21.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:20 smithi047 ceph-mon[30375]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:21.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:20 smithi064 ceph-mon[34537]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:21.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:20 smithi050 ceph-mon[34072]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:21.570 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:24:21.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:24:21.825 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:23.272 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:23 smithi047 ceph-mon[30375]: pgmap v393: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-11T12:24:23.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:23 smithi064 ceph-mon[34537]: pgmap v393: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-11T12:24:23.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:23 smithi050 ceph-mon[34072]: pgmap v393: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-05-11T12:24:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 -- 172.21.15.47:0/3546488062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 msgr2=0x7fd3381099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 --2- 172.21.15.47:0/3546488062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd3381099a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd32800b0a0 tx=0x7fd32800b3b0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 -- 172.21.15.47:0/3546488062 shutdown_connections 2024-05-11T12:24:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 --2- 172.21.15.47:0/3546488062 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd33806c640 0x7fd3381103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 --2- 172.21.15.47:0/3546488062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd3381099a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 --2- 172.21.15.47:0/3546488062 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33806bc80 0x7fd33806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 -- 172.21.15.47:0/3546488062 >> 172.21.15.47:0/3546488062 conn(0x7fd338079400 msgr2=0x7fd338079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:23.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 -- 172.21.15.47:0/3546488062 shutdown_connections 2024-05-11T12:24:23.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.547+0000 7fd33ec28700 1 -- 172.21.15.47:0/3546488062 wait complete. 2024-05-11T12:24:23.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 Processor -- start 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 -- start start 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33806bc80 0x7fd33819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd33806c640 0x7fd33819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd338199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd338113770 con 0x7fd3381095c0 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3381135f0 con 0x7fd33806c640 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33ec28700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3381138f0 con 0x7fd33806bc80 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.548+0000 7fd33d1c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd338199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd338199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46864/0 (socket says 172.21.15.47:46864) 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 -- 172.21.15.47:0/1558917848 learned_addr learned my addr 172.21.15.47:0/1558917848 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33c9c4700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33806bc80 0x7fd33819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:23.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd337fff700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd33806c640 0x7fd33819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 -- 172.21.15.47:0/1558917848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33806bc80 msgr2=0x7fd33819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33806bc80 0x7fd33819f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 -- 172.21.15.47:0/1558917848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd33806c640 msgr2=0x7fd33819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd33806c640 0x7fd33819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 -- 172.21.15.47:0/1558917848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33819a230 con 0x7fd3381095c0 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33d1c5700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd338199ad0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd33000b550 tx=0x7fd33000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd335ffb700 1 -- 172.21.15.47:0/1558917848 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330015020 con 0x7fd3381095c0 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd335ffb700 1 -- 172.21.15.47:0/1558917848 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd330011ce0 con 0x7fd3381095c0 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33819a520 con 0x7fd3381095c0 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.549+0000 7fd335ffb700 1 -- 172.21.15.47:0/1558917848 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330010c90 con 0x7fd3381095c0 2024-05-11T12:24:23.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.550+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3381a67e0 con 0x7fd3381095c0 2024-05-11T12:24:23.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.550+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd33810baa0 con 0x7fd3381095c0 2024-05-11T12:24:23.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.552+0000 7fd335ffb700 1 -- 172.21.15.47:0/1558917848 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd330018020 con 0x7fd3381095c0 2024-05-11T12:24:23.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.553+0000 7fd335ffb700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3240b2900 0x7fd3240b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:23.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.553+0000 7fd33c9c4700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3240b2900 0x7fd3240b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:23.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.553+0000 7fd335ffb700 1 -- 172.21.15.47:0/1558917848 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd3300e1e90 con 0x7fd3381095c0 2024-05-11T12:24:23.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.553+0000 7fd33c9c4700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3240b2900 0x7fd3240b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd328009e60 tx=0x7fd32800b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:23.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.556+0000 7fd335ffb700 1 -- 172.21.15.47:0/1558917848 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd33001f020 con 0x7fd3381095c0 2024-05-11T12:24:23.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.822+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd33819afc0 con 0x7fd3381095c0 2024-05-11T12:24:23.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.823+0000 7fd335ffb700 1 -- 172.21.15.47:0/1558917848 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fd3300a9ae0 con 0x7fd3381095c0 2024-05-11T12:24:23.823 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:24:23.823 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:24:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3240b2900 msgr2=0x7fd3240b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3240b2900 0x7fd3240b4dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd328009e60 tx=0x7fd32800b7b0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 msgr2=0x7fd338199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:23.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd338199ad0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd33000b550 tx=0x7fd33000b860 comp rx=0 tx=0).stop 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 shutdown_connections 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3381095c0 0x7fd338199ad0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd33806bc80 0x7fd33819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3240b2900 0x7fd3240b4dc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 --2- 172.21.15.47:0/1558917848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd33806c640 0x7fd33819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 >> 172.21.15.47:0/1558917848 conn(0x7fd338079400 msgr2=0x7fd3381011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 shutdown_connections 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:23.825+0000 7fd33ec28700 1 -- 172.21.15.47:0/1558917848 wait complete. 2024-05-11T12:24:23.826 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:24:24.159 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:24 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1558917848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:24.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:24 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1558917848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:24.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:24 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1558917848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:24:24.512 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:24:24.772 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:25.214 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:25 smithi047 ceph-mon[30375]: pgmap v394: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:25.350 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:25 smithi050 ceph-mon[34072]: pgmap v394: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:25.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:25 smithi064 ceph-mon[34537]: pgmap v394: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 -- 172.21.15.47:0/2488842644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5301041b0 msgr2=0x7ff5301081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 --2- 172.21.15.47:0/2488842644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5301041b0 0x7ff5301081d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff5240099c0 tx=0x7ff524009cd0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 -- 172.21.15.47:0/2488842644 shutdown_connections 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 --2- 172.21.15.47:0/2488842644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff530108900 0x7ff530110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 --2- 172.21.15.47:0/2488842644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff530103800 0x7ff530103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 --2- 172.21.15.47:0/2488842644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff5301041b0 0x7ff5301081d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 -- 172.21.15.47:0/2488842644 >> 172.21.15.47:0/2488842644 conn(0x7ff53007a6b0 msgr2=0x7ff53007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:26.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 -- 172.21.15.47:0/2488842644 shutdown_connections 2024-05-11T12:24:26.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.468+0000 7ff534b5d700 1 -- 172.21.15.47:0/2488842644 wait complete. 2024-05-11T12:24:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 Processor -- start 2024-05-11T12:24:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 -- start start 2024-05-11T12:24:26.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff530103800 0x7ff530075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5301041b0 0x7ff530075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff530108900 0x7ff5300762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301136c0 con 0x7ff530108900 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff530113540 con 0x7ff530103800 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff534b5d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff530113840 con 0x7ff5301041b0 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff530108900 0x7ff5300762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff530108900 0x7ff5300762d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46872/0 (socket says 172.21.15.47:46872) 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1241843732 learned_addr learned my addr 172.21.15.47:0/1241843732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5301041b0 0x7ff530075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52e59c700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff530103800 0x7ff530075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1241843732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5301041b0 msgr2=0x7ff530075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5301041b0 0x7ff530075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1241843732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff530103800 msgr2=0x7ff530075850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff530103800 0x7ff530075850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1241843732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5301a8140 con 0x7ff530108900 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.469+0000 7ff52e59c700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff530103800 0x7ff530075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.470+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff530108900 0x7ff5300762d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff528007ee0 tx=0x7ff52800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.470+0000 7ff51f7fe700 1 -- 172.21.15.47:0/1241843732 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52800ddd0 con 0x7ff530108900 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.470+0000 7ff51f7fe700 1 -- 172.21.15.47:0/1241843732 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff528004020 con 0x7ff530108900 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.470+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5301a8430 con 0x7ff530108900 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.470+0000 7ff51f7fe700 1 -- 172.21.15.47:0/1241843732 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528015660 con 0x7ff530108900 2024-05-11T12:24:26.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.470+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5301a8940 con 0x7ff530108900 2024-05-11T12:24:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.472+0000 7ff51f7fe700 1 -- 172.21.15.47:0/1241843732 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff5280041d0 con 0x7ff530108900 2024-05-11T12:24:26.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.473+0000 7ff51f7fe700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff5180b2900 0x7ff5180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff52e59c700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff5180b2900 0x7ff5180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff51f7fe700 1 -- 172.21.15.47:0/1241843732 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff5280e1cb0 con 0x7ff530108900 2024-05-11T12:24:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff53f3f3b80 1 Processor -- start 2024-05-11T12:24:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff52e59c700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff5180b2900 0x7ff5180b4dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff520005950 tx=0x7ff5200058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff53f3f3b80 1 -- start start 2024-05-11T12:24:26.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d93160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d8eb30 0x5569e8d8ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.474+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5569e8ce0940 con 0x5569e8d936a0 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5569e8c45da0 con 0x5569e8d92d80 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5569e8cdc900 con 0x5569e8d8eb30 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46878/0 (socket says 172.21.15.47:46878) 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3427401209 learned_addr learned my addr 172.21.15.47:0/3427401209 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d93160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d8eb30 0x5569e8d8ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3427401209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d8eb30 msgr2=0x5569e8d8ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d8eb30 0x5569e8d8ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3427401209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 msgr2=0x5569e8d93160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d93160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3427401209 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e8c2d8d0 con 0x5569e8d936a0 2024-05-11T12:24:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d8eb30 0x5569e8d8ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8e510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff524000c00 tx=0x7ff524009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5c6fefde008d32 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff51cff9700 1 -- 172.21.15.47:0/3427401209 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524003a60 con 0x5569e8d936a0 2024-05-11T12:24:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff51cff9700 1 -- 172.21.15.47:0/3427401209 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff524026070 con 0x5569e8d936a0 2024-05-11T12:24:26.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.475+0000 7ff51cff9700 1 -- 172.21.15.47:0/3427401209 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52400bb10 con 0x5569e8d936a0 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3427401209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 msgr2=0x5569e8d8e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8e510 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff524000c00 tx=0x7ff524009550 comp rx=0 tx=0).stop 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3427401209 shutdown_connections 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d8eb30 0x5569e8d8ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3427401209 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8e510 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3427401209 >> 172.21.15.47:0/3427401209 conn(0x5569e8cbab70 msgr2=0x5569e8d88fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3427401209 shutdown_connections 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.476+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3427401209 wait complete. 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 -- start start 2024-05-11T12:24:26.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 Processor -- start 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d8eb30 0x5569e8d9a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d977f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5569e8d9b210 con 0x5569e8d8eb30 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5569e8c99ed0 con 0x5569e8d92d80 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5569e8d9b390 con 0x5569e8d936a0 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff53f3f3b80 10 client.0 _ll_register_callbacks cb 0x5569e8c78950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff52e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d8eb30 0x5569e8d9a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff52e59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d8eb30 0x5569e8d9a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46886/0 (socket says 172.21.15.47:46886) 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.477+0000 7ff52e59c700 1 -- 172.21.15.47:0/3235039970 learned_addr learned my addr 172.21.15.47:0/3235039970 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52e59c700 1 -- 172.21.15.47:0/3235039970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 msgr2=0x5569e8d97d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d97d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52e59c700 1 -- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 msgr2=0x5569e8d977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52e59c700 1 -- 172.21.15.47:0/3235039970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e8c97760 con 0x5569e8d8eb30 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d8eb30 0x5569e8d9a450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff520004130 tx=0x7ff520004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd49fc09939c53e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff515ffb700 1 -- 172.21.15.47:0/3235039970 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520024020 con 0x5569e8d8eb30 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff515ffb700 1 -- 172.21.15.47:0/3235039970 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff520020440 con 0x5569e8d8eb30 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff524026070 con 0x5569e8d8eb30 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff515ffb700 1 -- 172.21.15.47:0/3235039970 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff52002f050 con 0x5569e8d8eb30 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff53f3f3b80 10 client.15165 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5569e8da5a90 con 0x5569e8d8eb30 2024-05-11T12:24:26.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.478+0000 7ff515ffb700 1 -- 172.21.15.47:0/3235039970 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff524026070 con 0x5569e8d8eb30 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff515ffb700 1 -- 172.21.15.47:0/3235039970 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff5200115d0 con 0x5569e8d8eb30 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff53f3f3b80 10 client.15165 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff53f3f3b80 10 client.15165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff53f3f3b80 20 client.15165 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8da8120 0x5569e8daa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff53f3f3b80 4 client.15165 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569e8cee320 con 0x5569e8da8120 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8da8120 0x5569e8daa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8da8120 0x5569e8daa550 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2e4d4ee0f38e7c server_cookie=c65ce65860bcf965 in_seq=0 out_seq=0 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.479+0000 7ff515ffb700 10 client.15165 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.480+0000 7ff515ffb700 1 -- 172.21.15.47:0/3235039970 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569e8cee320 con 0x5569e8da8120 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.480+0000 7ff515ffb700 10 client.15165 handle_command_reply: tid=0 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.480+0000 7ff53f3f3b80 1 client.15165 shutdown 2024-05-11T12:24:26.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.480+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d8eb30 msgr2=0x5569e8d9a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.480+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d8eb30 0x5569e8d9a450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff520004130 tx=0x7ff520004750 comp rx=0 tx=0).stop 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 shutdown_connections 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8da8120 0x5569e8daa550 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d97d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d8eb30 0x5569e8d9a450 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff515ffb700 0 client.15165 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3235039970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d92d80 0x5569e8d977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 >> 172.21.15.47:0/3235039970 conn(0x5569e8cbab70 msgr2=0x5569e8d83830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 shutdown_connections 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3235039970 wait complete. 2024-05-11T12:24:26.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.481+0000 7ff53f3f3b80 20 client.15165 trim_cache size 0 max 16384 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 Processor -- start 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 -- start start 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d92d80 0x5569e8d83940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301adf90 con 0x5569e8d936a0 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301b0810 con 0x5569e8d8eb30 2024-05-11T12:24:26.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301ae600 con 0x5569e8d92d80 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.496+0000 7ff52ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46900/0 (socket says 172.21.15.47:46900) 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1146760747 learned_addr learned my addr 172.21.15.47:0/1146760747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d92d80 0x5569e8d83940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52e59c700 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1146760747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d92d80 msgr2=0x5569e8d83940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d92d80 0x5569e8d83940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1146760747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 msgr2=0x5569e8da42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 -- 172.21.15.47:0/1146760747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e8d89d60 con 0x5569e8d936a0 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8dc10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff5280df680 tx=0x7ff5280a5190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101e08700700128f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff517fff700 1 -- 172.21.15.47:0/1146760747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280a91a0 con 0x5569e8d936a0 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff517fff700 1 -- 172.21.15.47:0/1146760747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5280c3070 con 0x5569e8d936a0 2024-05-11T12:24:26.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.497+0000 7ff517fff700 1 -- 172.21.15.47:0/1146760747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280a9400 con 0x5569e8d936a0 2024-05-11T12:24:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/1146760747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 msgr2=0x5569e8d8dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8dc10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff5280df680 tx=0x7ff5280a5190 comp rx=0 tx=0).stop 2024-05-11T12:24:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/1146760747 shutdown_connections 2024-05-11T12:24:26.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d936a0 0x5569e8d8dc10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/1146760747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d92d80 0x5569e8d83940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/1146760747 >> 172.21.15.47:0/1146760747 conn(0x5569e8cbab70 msgr2=0x5569e8d8d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/1146760747 shutdown_connections 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/1146760747 wait complete. 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- start start 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 Processor -- start 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d92d80 0x5569e8d94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d94640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301aecd0 con 0x5569e8d92d80 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301b0140 con 0x5569e8d8eb30 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.498+0000 7ff53f3f3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff5301af3a0 con 0x5569e8d936a0 2024-05-11T12:24:26.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff53f3f3b80 10 client.0 _ll_register_callbacks cb 0x5569e8c78950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d92d80 0x5569e8d94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d92d80 0x5569e8d94100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46916/0 (socket says 172.21.15.47:46916) 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3917510860 learned_addr learned my addr 172.21.15.47:0/3917510860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d94640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3917510860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 msgr2=0x5569e8d94640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d94640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 msgr2=0x5569e8da0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 -- 172.21.15.47:0/3917510860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e8d94bb0 con 0x5569e8d92d80 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52ed9d700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff52dd9b700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d92d80 0x5569e8d94100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff524006010 tx=0x7ff524009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f55a3b3c1888af4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff515ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:26.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff515ffb700 1 -- 172.21.15.47:0/3917510860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240052c0 con 0x5569e8d92d80 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff515ffb700 1 -- 172.21.15.47:0/3917510860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff524005460 con 0x5569e8d92d80 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5280c3070 con 0x5569e8d92d80 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff515ffb700 1 -- 172.21.15.47:0/3917510860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff524005660 con 0x5569e8d92d80 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff53f3f3b80 10 client.15171 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.499+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5569e8e12790 con 0x5569e8d92d80 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff515ffb700 1 -- 172.21.15.47:0/3917510860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5280c3070 con 0x5569e8d92d80 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff515ffb700 1 -- 172.21.15.47:0/3917510860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff52400f740 con 0x5569e8d92d80 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff53f3f3b80 10 client.15171 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff53f3f3b80 10 client.15171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff53f3f3b80 20 client.15171 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8e12a70 0x5569e8e14e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff53f3f3b80 4 client.15171 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5569e8ce83c0 con 0x5569e8e12a70 2024-05-11T12:24:26.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.500+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8e12a70 0x5569e8e14e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff52e59c700 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8e12a70 0x5569e8e14e60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77c3232279649290 server_cookie=31e312491dff5a92 in_seq=0 out_seq=0 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff515ffb700 10 client.15171 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff515ffb700 1 -- 172.21.15.47:0/3917510860 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5569e8ce83c0 con 0x5569e8e12a70 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff515ffb700 10 client.15171 handle_command_reply: tid=0 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 client.15171 shutdown 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d92d80 msgr2=0x5569e8d94100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d92d80 0x5569e8d94100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff524006010 tx=0x7ff524009310 comp rx=0 tx=0).stop 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 shutdown_connections 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5569e8e12a70 0x5569e8e14e60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5569e8d936a0 0x5569e8d94640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5569e8d8eb30 0x5569e8da0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff515ffb700 0 client.15171 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 --2- 172.21.15.47:0/3917510860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5569e8d92d80 0x5569e8d94100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 >> 172.21.15.47:0/3917510860 conn(0x5569e8cbab70 msgr2=0x5569e8d8dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 shutdown_connections 2024-05-11T12:24:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 1 -- 172.21.15.47:0/3917510860 wait complete. 2024-05-11T12:24:26.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.501+0000 7ff53f3f3b80 20 client.15171 trim_cache size 0 max 16384 2024-05-11T12:24:26.507 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:24:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff5180b2900 msgr2=0x7ff5180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff5180b2900 0x7ff5180b4dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff520005950 tx=0x7ff5200058e0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff530108900 msgr2=0x7ff5300762d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff530108900 0x7ff5300762d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff528007ee0 tx=0x7ff52800d3b0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 shutdown_connections 2024-05-11T12:24:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff530108900 0x7ff5300762d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff5180b2900 0x7ff5180b4dc0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff530103800 0x7ff530075850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 --2- 172.21.15.47:0/1241843732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff5301041b0 0x7ff530075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 >> 172.21.15.47:0/1241843732 conn(0x7ff53007a6b0 msgr2=0x7ff530101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 shutdown_connections 2024-05-11T12:24:26.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:26.502+0000 7ff534b5d700 1 -- 172.21.15.47:0/1241843732 wait complete. 2024-05-11T12:24:27.213 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:27 smithi047 ceph-mon[30375]: pgmap v395: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:27.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:27 smithi064 ceph-mon[34537]: pgmap v395: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:27.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:27 smithi050 ceph-mon[34072]: pgmap v395: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:28.244 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:24:28.245 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:24:28.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:28 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:24:28.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:28 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:24:28.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:28 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:24:28.506 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:29 smithi047 ceph-mon[30375]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:29 smithi064 ceph-mon[34537]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:29 smithi050 ceph-mon[34072]: pgmap v396: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:29.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:30.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:30 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:24:30.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:30 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:24:30.177 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:30 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 -- 172.21.15.47:0/3183998153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c1095c0 msgr2=0x7f111c1099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 --2- 172.21.15.47:0/3183998153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c1095c0 0x7f111c1099a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f110c009990 tx=0x7f110c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 -- 172.21.15.47:0/3183998153 shutdown_connections 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 --2- 172.21.15.47:0/3183998153 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f111c06bbf0 0x7f111c1103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 --2- 172.21.15.47:0/3183998153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c1095c0 0x7f111c1099a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 --2- 172.21.15.47:0/3183998153 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111c06b230 0x7f111c06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 -- 172.21.15.47:0/3183998153 >> 172.21.15.47:0/3183998153 conn(0x7f111c079400 msgr2=0x7f111c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 -- 172.21.15.47:0/3183998153 shutdown_connections 2024-05-11T12:24:30.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.179+0000 7f11224f9700 1 -- 172.21.15.47:0/3183998153 wait complete. 2024-05-11T12:24:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 Processor -- start 2024-05-11T12:24:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 -- start start 2024-05-11T12:24:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111c06b230 0x7f111c102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c06bbf0 0x7f111c102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f111c1095c0 0x7f111c102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f111c113750 con 0x7f111c06bbf0 2024-05-11T12:24:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1135d0 con 0x7f111c06b230 2024-05-11T12:24:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f11224f9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1138d0 con 0x7f111c1095c0 2024-05-11T12:24:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c06bbf0 0x7f111c102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f111b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c06bbf0 0x7f111c102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46020/0 (socket says 172.21.15.47:46020) 2024-05-11T12:24:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.180+0000 7f111b7fe700 1 -- 172.21.15.47:0/1476354617 learned_addr learned my addr 172.21.15.47:0/1476354617 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:30.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f111bfff700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111c06b230 0x7f111c102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f1120a96700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f111c1095c0 0x7f111c102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f111b7fe700 1 -- 172.21.15.47:0/1476354617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f111c1095c0 msgr2=0x7f111c102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f111b7fe700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f111c1095c0 0x7f111c102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f111b7fe700 1 -- 172.21.15.47:0/1476354617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111c06b230 msgr2=0x7f111c102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f111b7fe700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111c06b230 0x7f111c102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f111b7fe700 1 -- 172.21.15.47:0/1476354617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c1a8230 con 0x7f111c06bbf0 2024-05-11T12:24:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f111b7fe700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c06bbf0 0x7f111c102a60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f111000d980 tx=0x7f111000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f11197fa700 1 -- 172.21.15.47:0/1476354617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1110016070 con 0x7f111c06bbf0 2024-05-11T12:24:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f11197fa700 1 -- 172.21.15.47:0/1476354617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1110004830 con 0x7f111c06bbf0 2024-05-11T12:24:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c1a8520 con 0x7f111c06bbf0 2024-05-11T12:24:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f11197fa700 1 -- 172.21.15.47:0/1476354617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100054c0 con 0x7f111c06bbf0 2024-05-11T12:24:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.181+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1a8ab0 con 0x7f111c06bbf0 2024-05-11T12:24:30.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.184+0000 7f11197fa700 1 -- 172.21.15.47:0/1476354617 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1110004220 con 0x7f111c06bbf0 2024-05-11T12:24:30.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.184+0000 7f11197fa700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11040b2900 0x7f11040b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.184+0000 7f111bfff700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11040b2900 0x7f11040b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.184+0000 7f11197fa700 1 -- 172.21.15.47:0/1476354617 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f11100e20d0 con 0x7f111c06bbf0 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 Processor -- start 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f111bfff700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11040b2900 0x7f11040b4dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f110c00b5c0 tx=0x7f110c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 -- start start 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6be00 0x5623b8a6c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6c720 0x5623b8a67590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a68010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5623b89b99c0 con 0x5623b8a6be00 2024-05-11T12:24:30.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5623b891ee20 con 0x5623b8a67bb0 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5623b89b5980 con 0x5623b8a6c720 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f111bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6be00 0x5623b8a6c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f111bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6be00 0x5623b8a6c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46036/0 (socket says 172.21.15.47:46036) 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f111bfff700 1 -- 172.21.15.47:0/2872453636 learned_addr learned my addr 172.21.15.47:0/2872453636 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.185+0000 7f1120a96700 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a68010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f111b7fe700 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6c720 0x5623b8a67590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f111bfff700 1 -- 172.21.15.47:0/2872453636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6c720 msgr2=0x5623b8a67590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f111bfff700 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6c720 0x5623b8a67590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f111bfff700 1 -- 172.21.15.47:0/2872453636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 msgr2=0x5623b8a68010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f111bfff700 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a68010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f111bfff700 1 -- 172.21.15.47:0/2872453636 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b8906950 con 0x5623b8a6be00 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f1120a96700 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a68010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f111bfff700 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6be00 0x5623b8a6c1e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f110c00ba60 tx=0x7f110c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6380f469c3d7c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:30.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f11027fc700 1 -- 172.21.15.47:0/2872453636 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c034070 con 0x5623b8a6be00 2024-05-11T12:24:30.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f11027fc700 1 -- 172.21.15.47:0/2872453636 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c003680 con 0x5623b8a6be00 2024-05-11T12:24:30.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.186+0000 7f11027fc700 1 -- 172.21.15.47:0/2872453636 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c02f920 con 0x5623b8a6be00 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2872453636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6be00 msgr2=0x5623b8a6c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6be00 0x5623b8a6c1e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f110c00ba60 tx=0x7f110c01f000 comp rx=0 tx=0).stop 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2872453636 shutdown_connections 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a68010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6be00 0x5623b8a6c1e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2872453636 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6c720 0x5623b8a67590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2872453636 >> 172.21.15.47:0/2872453636 conn(0x5623b8993bf0 msgr2=0x5623b8a62060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2872453636 shutdown_connections 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2872453636 wait complete. 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.187+0000 7f112cd8fb80 1 -- start start 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 1 Processor -- start 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a67bb0 0x5623b8a734f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6be00 0x5623b8a70900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a70e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5623b8a74290 con 0x5623b8a6c720 2024-05-11T12:24:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5623b8972f50 con 0x5623b8a6be00 2024-05-11T12:24:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5623b8a74410 con 0x5623b8a67bb0 2024-05-11T12:24:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f112cd8fb80 10 client.0 _ll_register_callbacks cb 0x5623b89519d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f1120a96700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a70e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a67bb0 0x5623b8a734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a67bb0 0x5623b8a734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:58038/0 (socket says 172.21.15.47:58038) 2024-05-11T12:24:30.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f1120a96700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a70e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46038/0 (socket says 172.21.15.47:46038) 2024-05-11T12:24:30.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 -- 172.21.15.47:0/1163077544 learned_addr learned my addr 172.21.15.47:0/1163077544 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:30.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111b7fe700 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6be00 0x5623b8a70900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 -- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6be00 msgr2=0x5623b8a70900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6be00 0x5623b8a70900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 -- 172.21.15.47:0/1163077544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 msgr2=0x5623b8a70e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a70e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.188+0000 7f111bfff700 1 -- 172.21.15.47:0/1163077544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b89707e0 con 0x5623b8a67bb0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f111bfff700 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a67bb0 0x5623b8a734f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f110c005f50 tx=0x7f110c02f580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aebd3cfd58ec1a35 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f10fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f10fb7fe700 1 -- 172.21.15.47:0/1163077544 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c03f060 con 0x5623b8a67bb0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c003680 con 0x5623b8a67bb0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f10fb7fe700 1 -- 172.21.15.47:0/1163077544 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f110c02f770 con 0x5623b8a67bb0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f10fb7fe700 1 -- 172.21.15.47:0/1163077544 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f110c034040 con 0x5623b8a67bb0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f112cd8fb80 10 client.24856 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5623b8a74290 con 0x5623b8a67bb0 2024-05-11T12:24:30.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.189+0000 7f10fb7fe700 1 -- 172.21.15.47:0/1163077544 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c003680 con 0x5623b8a67bb0 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f10fb7fe700 1 -- 172.21.15.47:0/1163077544 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f110c00f7a0 con 0x5623b8a67bb0 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f112cd8fb80 10 client.24856 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f112cd8fb80 10 client.24856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f112cd8fb80 20 client.24856 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8a80e00 0x5623b8a831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f112cd8fb80 4 client.24856 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b89c73a0 con 0x5623b8a80e00 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f111b7fe700 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8a80e00 0x5623b8a831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f111b7fe700 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8a80e00 0x5623b8a831f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7948aa833e455fd server_cookie=525c1c70f12f66be in_seq=0 out_seq=0 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.190+0000 7f10fb7fe700 10 client.24856 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f10fb7fe700 1 -- 172.21.15.47:0/1163077544 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5623b89c73a0 con 0x5623b8a80e00 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f10fb7fe700 10 client.24856 handle_command_reply: tid=0 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 client.24856 shutdown 2024-05-11T12:24:30.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a67bb0 msgr2=0x5623b8a734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a67bb0 0x5623b8a734f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f110c005f50 tx=0x7f110c02f580 comp rx=0 tx=0).stop 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 shutdown_connections 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8a80e00 0x5623b8a831f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a70e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a67bb0 0x5623b8a734f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f10fb7fe700 0 client.24856 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/1163077544 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6be00 0x5623b8a70900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 >> 172.21.15.47:0/1163077544 conn(0x5623b8993bf0 msgr2=0x5623b8a5c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 shutdown_connections 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.191+0000 7f112cd8fb80 1 -- 172.21.15.47:0/1163077544 wait complete. 2024-05-11T12:24:30.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.192+0000 7f112cd8fb80 20 client.24856 trim_cache size 0 max 16384 2024-05-11T12:24:30.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 Processor -- start 2024-05-11T12:24:30.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 -- start start 2024-05-11T12:24:30.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a67bb0 0x5623b8a7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a6b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6c720 0x5623b8a88820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1ae130 con 0x5623b8a67bb0 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b09b0 con 0x5623b8a6c720 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1ae7a0 con 0x5623b8a6be00 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.212+0000 7f111bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a67bb0 0x5623b8a7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a67bb0 0x5623b8a7d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46054/0 (socket says 172.21.15.47:46054) 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 -- 172.21.15.47:0/3947710609 learned_addr learned my addr 172.21.15.47:0/3947710609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111b7fe700 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a6b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f1120a96700 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6c720 0x5623b8a88820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 -- 172.21.15.47:0/3947710609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 msgr2=0x5623b8a6b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a6b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 -- 172.21.15.47:0/3947710609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6c720 msgr2=0x5623b8a88820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6c720 0x5623b8a88820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 -- 172.21.15.47:0/3947710609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b8a5ea00 con 0x5623b8a67bb0 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f111bfff700 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a67bb0 0x5623b8a7d350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f110c02f550 tx=0x7f110c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e816b107b329c2de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f1101ffb700 1 -- 172.21.15.47:0/3947710609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c03f040 con 0x5623b8a67bb0 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f1101ffb700 1 -- 172.21.15.47:0/3947710609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c00f910 con 0x5623b8a67bb0 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.213+0000 7f1101ffb700 1 -- 172.21.15.47:0/3947710609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c017560 con 0x5623b8a67bb0 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 -- 172.21.15.47:0/3947710609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a67bb0 msgr2=0x5623b8a7d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a67bb0 0x5623b8a7d350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f110c02f550 tx=0x7f110c00bea0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 -- 172.21.15.47:0/3947710609 shutdown_connections 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a6c720 0x5623b8a88820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a67bb0 0x5623b8a7d350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/3947710609 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a6b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 -- 172.21.15.47:0/3947710609 >> 172.21.15.47:0/3947710609 conn(0x5623b8993bf0 msgr2=0x5623b8a66e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:30.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 -- 172.21.15.47:0/3947710609 shutdown_connections 2024-05-11T12:24:30.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 -- 172.21.15.47:0/3947710609 wait complete. 2024-05-11T12:24:30.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.214+0000 7f112cd8fb80 1 -- start start 2024-05-11T12:24:30.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 1 Processor -- start 2024-05-11T12:24:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a79a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a77e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a78360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1aee70 con 0x5623b8a6c720 2024-05-11T12:24:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b02e0 con 0x5623b8a67bb0 2024-05-11T12:24:30.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1af540 con 0x5623b8a6be00 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f112cd8fb80 10 client.0 _ll_register_callbacks cb 0x5623b89519d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a78360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a78360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46066/0 (socket says 172.21.15.47:46066) 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 -- 172.21.15.47:0/2914046380 learned_addr learned my addr 172.21.15.47:0/2914046380 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f111bfff700 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a79a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f111b7fe700 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a77e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 -- 172.21.15.47:0/2914046380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 msgr2=0x5623b8a77e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a77e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 -- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 msgr2=0x5623b8a79a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a79a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.215+0000 7f1120a96700 1 -- 172.21.15.47:0/2914046380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b8a78a20 con 0x5623b8a6c720 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f1120a96700 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a78360 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1114007a00 tx=0x7f1114007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f777f12ce9198dfc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f10fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f10fb7fe700 1 -- 172.21.15.47:0/2914046380 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111401c070 con 0x5623b8a6c720 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f10fb7fe700 1 -- 172.21.15.47:0/2914046380 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1114004bb0 con 0x5623b8a6c720 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110c00f910 con 0x5623b8a6c720 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f10fb7fe700 1 -- 172.21.15.47:0/2914046380 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1114004db0 con 0x5623b8a6c720 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 10 client.15189 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5623b8aec380 con 0x5623b8a6c720 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f10fb7fe700 1 -- 172.21.15.47:0/2914046380 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f110c00f910 con 0x5623b8a6c720 2024-05-11T12:24:30.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f10fb7fe700 1 -- 172.21.15.47:0/2914046380 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f111402aa60 con 0x5623b8a6c720 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 10 client.15189 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 10 client.15189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 20 client.15189 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8aeca70 0x5623b8aeee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 4 client.15189 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.216+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5623b89c1440 con 0x5623b8aeca70 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.217+0000 7f111bfff700 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8aeca70 0x5623b8aeee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.217+0000 7f111bfff700 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8aeca70 0x5623b8aeee90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e99ec89dc89257cc server_cookie=e7a1a5b2ac87fcd4 in_seq=0 out_seq=0 2024-05-11T12:24:30.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.217+0000 7f10fb7fe700 10 client.15189 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.218+0000 7f10fb7fe700 1 -- 172.21.15.47:0/2914046380 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5623b89c1440 con 0x5623b8aeca70 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.218+0000 7f10fb7fe700 10 client.15189 handle_command_reply: tid=0 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.218+0000 7f112cd8fb80 1 client.15189 shutdown 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 msgr2=0x5623b8a78360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a78360 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1114007a00 tx=0x7f1114007d10 comp rx=0 tx=0).stop 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 shutdown_connections 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5623b8aeca70 0x5623b8aeee90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5623b8a6c720 0x5623b8a78360 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5623b8a67bb0 0x5623b8a79a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 --2- 172.21.15.47:0/2914046380 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5623b8a6be00 0x5623b8a77e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 >> 172.21.15.47:0/2914046380 conn(0x5623b8993bf0 msgr2=0x5623b8a6b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f10fb7fe700 0 client.15189 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 shutdown_connections 2024-05-11T12:24:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 1 -- 172.21.15.47:0/2914046380 wait complete. 2024-05-11T12:24:30.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f112cd8fb80 20 client.15189 trim_cache size 0 max 16384 2024-05-11T12:24:30.225 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:24:30.226 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:24:30.226 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "4fb447e5-67be-4a98-b659-66cbdc78d509", 2024-05-11T12:24:30.226 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:24:30.226 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11040b2900 msgr2=0x7f11040b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f11224f9700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11040b2900 0x7f11040b4dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f110c00b5c0 tx=0x7f110c005fd0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c06bbf0 msgr2=0x7f111c102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.219+0000 7f11224f9700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c06bbf0 0x7f111c102a60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f111000d980 tx=0x7f111000dc90 comp rx=0 tx=0).stop 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 shutdown_connections 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f111c1095c0 0x7f111c102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11040b2900 0x7f11040b4dc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f111c06b230 0x7f111c102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 --2- 172.21.15.47:0/1476354617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f111c06bbf0 0x7f111c102a60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:30.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 >> 172.21.15.47:0/1476354617 conn(0x7f111c079400 msgr2=0x7f111c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:30.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 shutdown_connections 2024-05-11T12:24:30.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:30.220+0000 7f11224f9700 1 -- 172.21.15.47:0/1476354617 wait complete. 2024-05-11T12:24:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:30 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:24:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:30 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:24:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:30 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:30 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:24:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:30 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:24:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:30 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:24:30.921 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:24:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:24:31.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:31 smithi047 ceph-mon[30375]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:31.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:31 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:24:31.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:31 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:31.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:31 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:24:31.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:31 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:31 smithi064 ceph-mon[34537]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:31 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:24:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:31 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:31 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:24:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:31 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:31.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:31 smithi050 ceph-mon[34072]: pgmap v397: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:24:31.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:31 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:24:31.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:31 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:31.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:31 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:24:31.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:31 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:24:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1542208754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 msgr2=0x7f3eb010f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1542208754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb010f300 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800b0a0 tx=0x7f3ea800b3b0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1542208754 shutdown_connections 2024-05-11T12:24:32.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1542208754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb010f300 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1542208754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0108530 0x7f3eb0108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1542208754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3eb0102450 0x7f3eb01028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1542208754 >> 172.21.15.47:0/1542208754 conn(0x7f3eb0079400 msgr2=0x7f3eb0079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1542208754 shutdown_connections 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.980+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1542208754 wait complete. 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 Processor -- start 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 -- start start 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3eb0102450 0x7f3eb019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb019b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0108530 0x7f3eb019b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0112600 con 0x7f3eb0102e10 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0112480 con 0x7f3eb0108530 2024-05-11T12:24:32.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.981+0000 7f3eb4a36700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0112780 con 0x7f3eb0102450 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb019b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb019b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46090/0 (socket says 172.21.15.47:46090) 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 -- 172.21.15.47:0/1508747756 learned_addr learned my addr 172.21.15.47:0/1508747756 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3eae59c700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3eb0102450 0x7f3eb019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0108530 0x7f3eb019b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 -- 172.21.15.47:0/1508747756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3eb0102450 msgr2=0x7f3eb019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3eb0102450 0x7f3eb019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 -- 172.21.15.47:0/1508747756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0108530 msgr2=0x7f3eb019b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0108530 0x7f3eb019b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 -- 172.21.15.47:0/1508747756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb01a5eb0 con 0x7f3eb0102e10 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb019b000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800d980 tx=0x7f3e9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea7fff700 1 -- 172.21.15.47:0/1508747756 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98017070 con 0x7f3eb0102e10 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb01a61a0 con 0x7f3eb0102e10 2024-05-11T12:24:32.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea7fff700 1 -- 172.21.15.47:0/1508747756 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e98005020 con 0x7f3eb0102e10 2024-05-11T12:24:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3ea7fff700 1 -- 172.21.15.47:0/1508747756 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98009b50 con 0x7f3eb0102e10 2024-05-11T12:24:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.982+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb01a6690 con 0x7f3eb0102e10 2024-05-11T12:24:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.985+0000 7f3ea7fff700 1 -- 172.21.15.47:0/1508747756 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3e980051d0 con 0x7f3eb0102e10 2024-05-11T12:24:32.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.986+0000 7f3ea7fff700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3e9c0b2880 0x7f3e9c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.986+0000 7f3eae59c700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3e9c0b2880 0x7f3e9c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.986+0000 7f3ea7fff700 1 -- 172.21.15.47:0/1508747756 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f3e980e2210 con 0x7f3eb0102e10 2024-05-11T12:24:32.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.986+0000 7f3ebf2ccb80 1 Processor -- start 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.986+0000 7f3eae59c700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3e9c0b2880 0x7f3e9c0b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0005950 tx=0x7f3ea000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.986+0000 7f3ebf2ccb80 1 -- start start 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acba0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e4acaf2940 con 0x55e4acba0b30 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e4aca57da0 con 0x55e4acba4d80 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e4acaee900 con 0x55e4acba56a0 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acba0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acba0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46106/0 (socket says 172.21.15.47:46106) 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 -- 172.21.15.47:0/2845243587 learned_addr learned my addr 172.21.15.47:0/2845243587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eae59c700 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 -- 172.21.15.47:0/2845243587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 msgr2=0x55e4acba0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:32.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 -- 172.21.15.47:0/2845243587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 msgr2=0x55e4acba5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 -- 172.21.15.47:0/2845243587 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4aca3f8d0 con 0x55e4acba0b30 2024-05-11T12:24:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.987+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acba0f90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80090d0 tx=0x7f3ea8009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da22ddd9e59e1f42 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ea4d99700 1 -- 172.21.15.47:0/2845243587 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8004a50 con 0x55e4acba0b30 2024-05-11T12:24:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ea4d99700 1 -- 172.21.15.47:0/2845243587 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8004bf0 con 0x55e4acba0b30 2024-05-11T12:24:32.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ea4d99700 1 -- 172.21.15.47:0/2845243587 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea801c640 con 0x55e4acba0b30 2024-05-11T12:24:32.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/2845243587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 msgr2=0x55e4acba0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acba0f90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80090d0 tx=0x7f3ea8009100 comp rx=0 tx=0).stop 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/2845243587 shutdown_connections 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acba0f90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/2845243587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.988+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/2845243587 >> 172.21.15.47:0/2845243587 conn(0x55e4acaccb70 msgr2=0x55e4acb9afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/2845243587 shutdown_connections 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/2845243587 wait complete. 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 -- start start 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 Processor -- start 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acbac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e4acbad210 con 0x55e4acba0b30 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e4acaabed0 con 0x55e4acba4d80 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.989+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e4acbad390 con 0x55e4acba56a0 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3ebf2ccb80 10 client.0 _ll_register_callbacks cb 0x55e4aca8a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acbac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acbac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46108/0 (socket says 172.21.15.47:46108) 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 -- 172.21.15.47:0/3650749150 learned_addr learned my addr 172.21.15.47:0/3650749150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 -- 172.21.15.47:0/3650749150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 msgr2=0x55e4acba9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 -- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 msgr2=0x55e4acba97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 -- 172.21.15.47:0/3650749150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4acaa9760 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3eae59c700 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acbac440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ea001da60 tx=0x7f3ea00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce5f898c8b3fcc6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3e95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3e95ffb700 1 -- 172.21.15.47:0/3650749150 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0016070 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3e95ffb700 1 -- 172.21.15.47:0/3650749150 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f3ea0004a30 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea8004bf0 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.990+0000 7f3e95ffb700 1 -- 172.21.15.47:0/3650749150 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3ea0004c70 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 10 client.15201 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e4acbb7a00 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3e95ffb700 1 -- 172.21.15.47:0/3650749150 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8004bf0 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3e95ffb700 1 -- 172.21.15.47:0/3650749150 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3ea0010970 con 0x55e4acba0b30 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 10 client.15201 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 10 client.15201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 20 client.15201 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acbba090 0x55e4acbbc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:32.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 4 client.15201 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4acb00320 con 0x55e4acbba090 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.991+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acbba090 0x55e4acbbc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.992+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acbba090 0x55e4acbbc4f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb63a4bec831341 server_cookie=58eab7254f928325 in_seq=0 out_seq=0 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.992+0000 7f3e95ffb700 10 client.15201 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.992+0000 7f3e95ffb700 1 -- 172.21.15.47:0/3650749150 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e4acb00320 con 0x55e4acbba090 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.992+0000 7f3e95ffb700 10 client.15201 handle_command_reply: tid=0 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.992+0000 7f3ebf2ccb80 1 client.15201 shutdown 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 msgr2=0x55e4acbac440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acbac440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ea001da60 tx=0x7f3ea00047c0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 shutdown_connections 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acbba090 0x55e4acbbc4f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acbac440 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3e95ffb700 0 client.15201 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3650749150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 >> 172.21.15.47:0/3650749150 conn(0x55e4acaccb70 msgr2=0x55e4acba63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 shutdown_connections 2024-05-11T12:24:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3650749150 wait complete. 2024-05-11T12:24:32.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:32.993+0000 7f3ebf2ccb80 20 client.15201 trim_cache size 0 max 16384 2024-05-11T12:24:33.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 Processor -- start 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 -- start start 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba0b30 0x55e4acba3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acb95940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba56a0 0x55e4acbb2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01abd90 con 0x55e4acba56a0 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01ae610 con 0x55e4acba4d80 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01ac400 con 0x55e4acba0b30 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3eaed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba56a0 0x55e4acbb2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3eaed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba56a0 0x55e4acbb2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46114/0 (socket says 172.21.15.47:46114) 2024-05-11T12:24:33.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.013+0000 7f3eaed9d700 1 -- 172.21.15.47:0/3448964372 learned_addr learned my addr 172.21.15.47:0/3448964372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3eae59c700 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba0b30 0x55e4acba3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acb95940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3eaed9d700 1 -- 172.21.15.47:0/3448964372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba0b30 msgr2=0x55e4acba3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba0b30 0x55e4acba3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3eaed9d700 1 -- 172.21.15.47:0/3448964372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 msgr2=0x55e4acb95940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acb95940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3eaed9d700 1 -- 172.21.15.47:0/3448964372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4acb97980 con 0x55e4acba56a0 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba56a0 0x55e4acbb2a00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800b830 tx=0x7f3ea800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32a8b36ef8bd5cf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3e97fff700 1 -- 172.21.15.47:0/3448964372 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8009320 con 0x55e4acba56a0 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3e97fff700 1 -- 172.21.15.47:0/3448964372 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8007950 con 0x55e4acba56a0 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.014+0000 7f3e97fff700 1 -- 172.21.15.47:0/3448964372 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea801cb30 con 0x55e4acba56a0 2024-05-11T12:24:33.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3448964372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba56a0 msgr2=0x55e4acbb2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba56a0 0x55e4acbb2a00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800b830 tx=0x7f3ea800b8e0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3448964372 shutdown_connections 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba56a0 0x55e4acbb2a00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba0b30 0x55e4acba3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/3448964372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acb95940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3448964372 >> 172.21.15.47:0/3448964372 conn(0x55e4acaccb70 msgr2=0x55e4acba9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3448964372 shutdown_connections 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/3448964372 wait complete. 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 -- start start 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 Processor -- start 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.015+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acb9ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:33.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ebf2ccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01acad0 con 0x55e4acba0b30 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01adf40 con 0x55e4acba4d80 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ebf2ccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb01ad1a0 con 0x55e4acba56a0 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ebf2ccb80 10 client.0 _ll_register_callbacks cb 0x55e4aca8a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acb9ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acb9ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46116/0 (socket says 172.21.15.47:46116) 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 -- 172.21.15.47:0/884609860 learned_addr learned my addr 172.21.15.47:0/884609860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:24:33.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eaed9d700 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 -- 172.21.15.47:0/884609860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 msgr2=0x55e4acba6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 -- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 msgr2=0x55e4acba6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:24:33.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 -- 172.21.15.47:0/884609860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4acba7810 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3e95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3eae59c700 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acb9ecb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0010010 tx=0x7f3ea0004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a780e40f5ffae2e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3e95ffb700 1 -- 172.21.15.47:0/884609860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0034040 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3e95ffb700 1 -- 172.21.15.47:0/884609860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f3ea0027bb0 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.016+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ea8007950 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3e95ffb700 1 -- 172.21.15.47:0/884609860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3ea0027db0 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 10 client.15207 fetch_fsmap learned FSMap version 20 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e4acc29a70 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3e95ffb700 1 -- 172.21.15.47:0/884609860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea8007950 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3e95ffb700 1 -- 172.21.15.47:0/884609860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3ea0005340 con 0x55e4acba0b30 2024-05-11T12:24:33.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 10 client.15207 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 10 client.15207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 20 client.15207 populate_metadata read hostname 'smithi047' 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acc2a180 0x55e4acc2c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 4 client.15207 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4acafa3c0 con 0x55e4acc2a180 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.017+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acc2a180 0x55e4acc2c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3e95ffb700 10 client.15207 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ea5d9b700 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acc2a180 0x55e4acc2c5a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a3f15947e4e525 server_cookie=126feb7d19db5db2 in_seq=0 out_seq=0 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3e95ffb700 1 -- 172.21.15.47:0/884609860 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e4acafa3c0 con 0x55e4acc2a180 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3e95ffb700 10 client.15207 handle_command_reply: tid=0 2024-05-11T12:24:33.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 client.15207 shutdown 2024-05-11T12:24:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 msgr2=0x55e4acb9ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acb9ecb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0010010 tx=0x7f3ea0004ae0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 shutdown_connections 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e4acc2a180 0x55e4acc2c5a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e4acba56a0 0x55e4acba6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e4acba0b30 0x55e4acb9ecb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 --2- 172.21.15.47:0/884609860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e4acba4d80 0x55e4acba6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3e95ffb700 0 client.15207 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 >> 172.21.15.47:0/884609860 conn(0x55e4acaccb70 msgr2=0x55e4acb94930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 shutdown_connections 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.018+0000 7f3ebf2ccb80 1 -- 172.21.15.47:0/884609860 wait complete. 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3ebf2ccb80 20 client.15207 trim_cache size 0 max 16384 2024-05-11T12:24:33.026 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "4fb447e5-67be-4a98-b659-66cbdc78d509": { 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "4fb447e5-67be-4a98-b659-66cbdc78d509", 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:24:33.027 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:24:33.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3e9c0b2880 msgr2=0x7f3e9c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:33.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3e9c0b2880 0x7f3e9c0b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3ea0005950 tx=0x7f3ea000a300 comp rx=0 tx=0).stop 2024-05-11T12:24:33.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 msgr2=0x7f3eb019b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:24:33.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb019b000 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800d980 tx=0x7f3e9800dc90 comp rx=0 tx=0).stop 2024-05-11T12:24:33.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 shutdown_connections 2024-05-11T12:24:33.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3e9c0b2880 0x7f3e9c0b4d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3eb0108530 0x7f3eb019b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3eb0102450 0x7f3eb019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 --2- 172.21.15.47:0/1508747756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3eb0102e10 0x7f3eb019b000 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:24:33.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 >> 172.21.15.47:0/1508747756 conn(0x7f3eb0079400 msgr2=0x7f3eb01004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:24:33.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 shutdown_connections 2024-05-11T12:24:33.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:24:33.019+0000 7f3eb4a36700 1 -- 172.21.15.47:0/1508747756 wait complete. 2024-05-11T12:24:33.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:33 smithi047 ceph-mon[30375]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 511 B/s wr, 2 op/s 2024-05-11T12:24:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:33 smithi064 ceph-mon[34537]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 511 B/s wr, 2 op/s 2024-05-11T12:24:33.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:33 smithi050 ceph-mon[34072]: pgmap v398: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 511 B/s wr, 2 op/s 2024-05-11T12:24:33.765 INFO:tasks.cephfs.filesystem:scrub status for tag:4fb447e5-67be-4a98-b659-66cbdc78d509 - {'path': '/', 'tag': '4fb447e5-67be-4a98-b659-66cbdc78d509', 'options': 'recursive,force'} 2024-05-11T12:24:34.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:34 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:24:34.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:34 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:24:34.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:34 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:24:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:35 smithi064 ceph-mon[34537]: pgmap v399: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:35.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:35 smithi050 ceph-mon[34072]: pgmap v399: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:35.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:35 smithi047 ceph-mon[30375]: pgmap v399: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:37 smithi064 ceph-mon[34537]: pgmap v400: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:37.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:37 smithi050 ceph-mon[34072]: pgmap v400: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:37.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:37 smithi047 ceph-mon[30375]: pgmap v400: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:39 smithi064 ceph-mon[34537]: pgmap v401: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:39.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:39 smithi050 ceph-mon[34072]: pgmap v401: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:39.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:39 smithi047 ceph-mon[30375]: pgmap v401: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:41.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:41 smithi064 ceph-mon[34537]: pgmap v402: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:41.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:41 smithi050 ceph-mon[34072]: pgmap v402: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:41.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:41 smithi047 ceph-mon[30375]: pgmap v402: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:43.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:43 smithi064 ceph-mon[34537]: pgmap v403: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:43.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:43 smithi050 ceph-mon[34072]: pgmap v403: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:43.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:43 smithi047 ceph-mon[30375]: pgmap v403: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:45.327 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:45 smithi047 ceph-mon[30375]: pgmap v404: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:45 smithi064 ceph-mon[34537]: pgmap v404: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:45.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:45 smithi050 ceph-mon[34072]: pgmap v404: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:24:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:47 smithi064 ceph-mon[34537]: pgmap v405: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:47.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:47 smithi050 ceph-mon[34072]: pgmap v405: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:47.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:47 smithi047 ceph-mon[30375]: pgmap v405: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:24:49.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:49 smithi064 ceph-mon[34537]: pgmap v406: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:49.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:49 smithi050 ceph-mon[34072]: pgmap v406: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:49.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:49 smithi047 ceph-mon[30375]: pgmap v406: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:24:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:51 smithi064 ceph-mon[34537]: pgmap v407: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-05-11T12:24:51.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:51 smithi050 ceph-mon[34072]: pgmap v407: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-05-11T12:24:51.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:51 smithi047 ceph-mon[30375]: pgmap v407: 97 pgs: 97 active+clean; 649 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-05-11T12:24:53.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:53 smithi064 ceph-mon[34537]: pgmap v408: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:53.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:53 smithi050 ceph-mon[34072]: pgmap v408: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:53.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:53 smithi047 ceph-mon[30375]: pgmap v408: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:55 smithi064 ceph-mon[34537]: pgmap v409: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:55.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:55 smithi050 ceph-mon[34072]: pgmap v409: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:55.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:55 smithi047 ceph-mon[30375]: pgmap v409: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:57.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:57 smithi064 ceph-mon[34537]: pgmap v410: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:57.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:57 smithi050 ceph-mon[34072]: pgmap v410: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:57.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:57 smithi047 ceph-mon[30375]: pgmap v410: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:24:59 smithi064 ceph-mon[34537]: pgmap v411: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:59.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:24:59 smithi050 ceph-mon[34072]: pgmap v411: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:24:59.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:24:59 smithi047 ceph-mon[30375]: pgmap v411: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:25:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:01 smithi064 ceph-mon[34537]: pgmap v412: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:25:01.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:01 smithi050 ceph-mon[34072]: pgmap v412: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:25:01.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:01 smithi047 ceph-mon[30375]: pgmap v412: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-05-11T12:25:03.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:03 smithi064 ceph-mon[34537]: pgmap v413: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:25:03.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:03 smithi050 ceph-mon[34072]: pgmap v413: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:25:03.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:03 smithi047 ceph-mon[30375]: pgmap v413: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 597 B/s wr, 2 op/s 2024-05-11T12:25:03.766 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:25:04.050 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:25:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:05 smithi064 ceph-mon[34537]: pgmap v414: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:05.434 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:05 smithi047 ceph-mon[30375]: pgmap v414: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:05 smithi050 ceph-mon[34072]: pgmap v414: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 -- 172.21.15.47:0/976482699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 msgr2=0x7f0c841103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/976482699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841103d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009990 tx=0x7f0c7c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 -- 172.21.15.47:0/976482699 shutdown_connections 2024-05-11T12:25:06.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/976482699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841103d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/976482699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c841095b0 0x7f0c84109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/976482699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c8406bac0 0x7f0c8406bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 -- 172.21.15.47:0/976482699 >> 172.21.15.47:0/976482699 conn(0x7f0c840793c0 msgr2=0x7f0c840797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 -- 172.21.15.47:0/976482699 shutdown_connections 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.013+0000 7f0c8b26d700 1 -- 172.21.15.47:0/976482699 wait complete. 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 Processor -- start 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 -- start start 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c8406bac0 0x7f0c841a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841095b0 0x7f0c8410f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841136d0 con 0x7f0c8406c480 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84113550 con 0x7f0c8406bac0 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.014+0000 7f0c8b26d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84113850 con 0x7f0c841095b0 2024-05-11T12:25:06.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c89009700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c8406bac0 0x7f0c841a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841a5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42492/0 (socket says 172.21.15.47:42492) 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 -- 172.21.15.47:0/1374744578 learned_addr learned my addr 172.21.15.47:0/1374744578 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c8980a700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841095b0 0x7f0c8410f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 -- 172.21.15.47:0/1374744578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841095b0 msgr2=0x7f0c8410f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841095b0 0x7f0c8410f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 -- 172.21.15.47:0/1374744578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c8406bac0 msgr2=0x7f0c841a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c8406bac0 0x7f0c841a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 -- 172.21.15.47:0/1374744578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8410fd10 con 0x7f0c8406c480 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c88808700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841a5300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400d980 tx=0x7f0c7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c7a7fc700 1 -- 172.21.15.47:0/1374744578 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74016070 con 0x7f0c8406c480 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c7a7fc700 1 -- 172.21.15.47:0/1374744578 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c74004830 con 0x7f0c8406c480 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c84110000 con 0x7f0c8406c480 2024-05-11T12:25:06.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c7a7fc700 1 -- 172.21.15.47:0/1374744578 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74005540 con 0x7f0c8406c480 2024-05-11T12:25:06.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.015+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841accf0 con 0x7f0c8406c480 2024-05-11T12:25:06.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.018+0000 7f0c7a7fc700 1 -- 172.21.15.47:0/1374744578 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0c74004220 con 0x7f0c8406c480 2024-05-11T12:25:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.019+0000 7f0c7a7fc700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0c700b2900 0x7f0c700b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.019+0000 7f0c89009700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0c700b2900 0x7f0c700b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.019+0000 7f0c7a7fc700 1 -- 172.21.15.47:0/1374744578 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0c740e2030 con 0x7f0c8406c480 2024-05-11T12:25:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.019+0000 7f0c95b03b80 1 Processor -- start 2024-05-11T12:25:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.019+0000 7f0c95b03b80 1 -- start start 2024-05-11T12:25:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.019+0000 7f0c89009700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0c700b2900 0x7f0c700b4dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009550 tx=0x7f0c800091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:06.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1e6a0 0x55a151d19510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a151c6b940 con 0x55a151d19b30 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a151bd0da0 con 0x55a151d1dd80 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a151c67900 con 0x55a151d1e6a0 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c8980a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c8980a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d19f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42496/0 (socket says 172.21.15.47:42496) 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c8980a700 1 -- 172.21.15.47:0/4191005199 learned_addr learned my addr 172.21.15.47:0/4191005199 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c88808700 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1e6a0 0x55a151d19510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c89009700 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d1e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c88808700 1 -- 172.21.15.47:0/4191005199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 msgr2=0x55a151d1e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c88808700 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d1e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c88808700 1 -- 172.21.15.47:0/4191005199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 msgr2=0x55a151d19f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c88808700 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d19f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c88808700 1 -- 172.21.15.47:0/4191005199 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a151bb88d0 con 0x55a151d1e6a0 2024-05-11T12:25:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c8980a700 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:25:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c89009700 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:25:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c88808700 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1e6a0 0x55a151d19510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c74000c00 tx=0x7f0c740a9160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=924496b660f555d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.020+0000 7f0c6f7fe700 1 -- 172.21.15.47:0/4191005199 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c740ad930 con 0x55a151d1e6a0 2024-05-11T12:25:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c6f7fe700 1 -- 172.21.15.47:0/4191005199 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c740b23e0 con 0x55a151d1e6a0 2024-05-11T12:25:06.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c6f7fe700 1 -- 172.21.15.47:0/4191005199 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c740a97e0 con 0x55a151d1e6a0 2024-05-11T12:25:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4191005199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1e6a0 msgr2=0x55a151d19510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1e6a0 0x55a151d19510 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0c74000c00 tx=0x7f0c740a9160 comp rx=0 tx=0).stop 2024-05-11T12:25:06.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4191005199 shutdown_connections 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d1e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4191005199 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1e6a0 0x55a151d19510 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4191005199 >> 172.21.15.47:0/4191005199 conn(0x55a151c45b70 msgr2=0x55a151d13fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4191005199 shutdown_connections 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.021+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4191005199 wait complete. 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 -- start start 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 Processor -- start 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d19b30 0x55a151d255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d22950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d1e6a0 0x55a151d22e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a151d26270 con 0x55a151d1e6a0 2024-05-11T12:25:06.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a151c24ed0 con 0x55a151d1dd80 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a151d263f0 con 0x55a151d19b30 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c95b03b80 10 client.0 _ll_register_callbacks cb 0x55a151c03950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c8980a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d1e6a0 0x55a151d22e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c8980a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d1e6a0 0x55a151d22e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42498/0 (socket says 172.21.15.47:42498) 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.022+0000 7f0c8980a700 1 -- 172.21.15.47:0/1383388526 learned_addr learned my addr 172.21.15.47:0/1383388526 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c89009700 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d19b30 0x55a151d255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c88808700 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d22950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c8980a700 1 -- 172.21.15.47:0/1383388526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d19b30 msgr2=0x55a151d255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c8980a700 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d19b30 0x55a151d255b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c8980a700 1 -- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 msgr2=0x55a151d22950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c8980a700 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d22950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c8980a700 1 -- 172.21.15.47:0/1383388526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a151c22760 con 0x55a151d1e6a0 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c8980a700 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d1e6a0 0x55a151d22e90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0059d0 tx=0x7f0c7c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00b0a12095ae0f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:25:06.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c6cff9700 1 -- 172.21.15.47:0/1383388526 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c003c70 con 0x55a151d1e6a0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c6cff9700 1 -- 172.21.15.47:0/1383388526 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0c7c005070 con 0x55a151d1e6a0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c740b23e0 con 0x55a151d1e6a0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c6cff9700 1 -- 172.21.15.47:0/1383388526 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0c7c031050 con 0x55a151d1e6a0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c95b03b80 10 client.15219 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.023+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a151d30b70 con 0x55a151d1e6a0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c6cff9700 1 -- 172.21.15.47:0/1383388526 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c740b23e0 con 0x55a151d1e6a0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c6cff9700 1 -- 172.21.15.47:0/1383388526 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0c7c00f7e0 con 0x55a151d1e6a0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c95b03b80 10 client.15219 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c95b03b80 10 client.15219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c95b03b80 20 client.15219 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d335b0 0x55a151d359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c95b03b80 4 client.15219 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a151c79320 con 0x55a151d335b0 2024-05-11T12:25:06.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.024+0000 7f0c89009700 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d335b0 0x55a151d359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c89009700 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d335b0 0x55a151d359d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d7c38737705aff server_cookie=c7bbd4e1da3e4b0c in_seq=0 out_seq=0 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c6cff9700 10 client.15219 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c6cff9700 1 -- 172.21.15.47:0/1383388526 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a151c79320 con 0x55a151d335b0 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c6cff9700 10 client.15219 handle_command_reply: tid=0 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 client.15219 shutdown 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d1e6a0 msgr2=0x55a151d22e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d1e6a0 0x55a151d22e90 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0059d0 tx=0x7f0c7c00bf60 comp rx=0 tx=0).stop 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 shutdown_connections 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d335b0 0x55a151d359d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d1e6a0 0x55a151d22e90 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d19b30 0x55a151d255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.025+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/1383388526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1dd80 0x55a151d22950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.026+0000 7f0c6cff9700 0 client.15219 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:06.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.026+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 >> 172.21.15.47:0/1383388526 conn(0x55a151c45b70 msgr2=0x55a151d1f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:06.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.026+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 shutdown_connections 2024-05-11T12:25:06.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.026+0000 7f0c95b03b80 1 -- 172.21.15.47:0/1383388526 wait complete. 2024-05-11T12:25:06.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.026+0000 7f0c95b03b80 20 client.15219 trim_cache size 0 max 16384 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 Processor -- start 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 -- start start 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d218b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d0e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d2f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b24b0 con 0x55a151d19b30 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b4d30 con 0x55a151d1e6a0 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b2b20 con 0x55a151d1dd80 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c89009700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c89009700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d218b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42512/0 (socket says 172.21.15.47:42512) 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c89009700 1 -- 172.21.15.47:0/3526868522 learned_addr learned my addr 172.21.15.47:0/3526868522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:06.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.046+0000 7f0c8980a700 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d2f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c88808700 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d0e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c89009700 1 -- 172.21.15.47:0/3526868522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 msgr2=0x55a151d0e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c89009700 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d0e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c89009700 1 -- 172.21.15.47:0/3526868522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 msgr2=0x55a151d2f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c89009700 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d2f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c89009700 1 -- 172.21.15.47:0/3526868522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a151d20ec0 con 0x55a151d19b30 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c8980a700 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c89009700 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d218b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c800048c0 tx=0x7f0c80013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b4117b12ff5a92 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c6effd700 1 -- 172.21.15.47:0/3526868522 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80030070 con 0x55a151d19b30 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c6effd700 1 -- 172.21.15.47:0/3526868522 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8001d040 con 0x55a151d19b30 2024-05-11T12:25:06.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.047+0000 7f0c6effd700 1 -- 172.21.15.47:0/3526868522 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8001c040 con 0x55a151d19b30 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 -- 172.21.15.47:0/3526868522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 msgr2=0x55a151d218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d218b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0c800048c0 tx=0x7f0c80013000 comp rx=0 tx=0).stop 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 -- 172.21.15.47:0/3526868522 shutdown_connections 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d218b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/3526868522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d0e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 -- 172.21.15.47:0/3526868522 >> 172.21.15.47:0/3526868522 conn(0x55a151c45b70 msgr2=0x55a151d19210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 -- 172.21.15.47:0/3526868522 shutdown_connections 2024-05-11T12:25:06.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 -- 172.21.15.47:0/3526868522 wait complete. 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 -- start start 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 Processor -- start 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.048+0000 7f0c95b03b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d9ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b31f0 con 0x55a151d19b30 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b4660 con 0x55a151d1e6a0 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c95b03b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841b38c0 con 0x55a151d1dd80 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c95b03b80 10 client.0 _ll_register_callbacks cb 0x55a151c03950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d278f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:42528/0 (socket says 172.21.15.47:42528) 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 -- 172.21.15.47:0/4263349770 learned_addr learned my addr 172.21.15.47:0/4263349770 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c88808700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c8980a700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d9ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 -- 172.21.15.47:0/4263349770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 msgr2=0x55a151d27e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d27e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 -- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 msgr2=0x55a151d9ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d9ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 -- 172.21.15.47:0/4263349770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a151d9d3b0 con 0x55a151d19b30 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c8980a700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d9ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c89009700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d278f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c8002aa20 tx=0x7f0c80007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473329eec3407a12 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c6cff9700 1 -- 172.21.15.47:0/4263349770 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8001c070 con 0x55a151d19b30 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c6cff9700 1 -- 172.21.15.47:0/4263349770 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0c80004cd0 con 0x55a151d19b30 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c8001d040 con 0x55a151d19b30 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.049+0000 7f0c6cff9700 1 -- 172.21.15.47:0/4263349770 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0c80030040 con 0x55a151d19b30 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 10 client.15225 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:06.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a151d9e670 con 0x55a151d19b30 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c6cff9700 1 -- 172.21.15.47:0/4263349770 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c8001d040 con 0x55a151d19b30 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c6cff9700 1 -- 172.21.15.47:0/4263349770 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0c80022040 con 0x55a151d19b30 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 10 client.15225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 20 client.15225 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d9ec10 0x55a151da1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 4 client.15225 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a151c733c0 con 0x55a151d9ec10 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.050+0000 7f0c88808700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d9ec10 0x55a151da1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c88808700 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d9ec10 0x55a151da1030 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=984c0247eb25e893 server_cookie=78f3514e78f27080 in_seq=0 out_seq=0 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c6cff9700 10 client.15225 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c6cff9700 1 -- 172.21.15.47:0/4263349770 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a151c733c0 con 0x55a151d9ec10 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c6cff9700 10 client.15225 handle_command_reply: tid=0 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 client.15225 shutdown 2024-05-11T12:25:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 msgr2=0x55a151d278f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d278f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0c8002aa20 tx=0x7f0c80007f50 comp rx=0 tx=0).stop 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 shutdown_connections 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a151d9ec10 0x55a151da1030 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a151d1e6a0 0x55a151d9ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a151d19b30 0x55a151d278f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 --2- 172.21.15.47:0/4263349770 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a151d1dd80 0x55a151d27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 >> 172.21.15.47:0/4263349770 conn(0x55a151c45b70 msgr2=0x55a151d1c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c6cff9700 0 client.15225 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 shutdown_connections 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 1 -- 172.21.15.47:0/4263349770 wait complete. 2024-05-11T12:25:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.051+0000 7f0c95b03b80 20 client.15225 trim_cache size 0 max 16384 2024-05-11T12:25:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:25:06.058 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:25:06.058 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:25:06.058 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:25:06.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0c700b2900 msgr2=0x7f0c700b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0c700b2900 0x7f0c700b4dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0c80009550 tx=0x7f0c800091b0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 msgr2=0x7f0c841a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841a5300 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400d980 tx=0x7f0c7400dc90 comp rx=0 tx=0).stop 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 shutdown_connections 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0c841095b0 0x7f0c8410f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0c700b2900 0x7f0c700b4dc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0c8406bac0 0x7f0c841a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 --2- 172.21.15.47:0/1374744578 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c8406c480 0x7f0c841a5300 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 >> 172.21.15.47:0/1374744578 conn(0x7f0c840793c0 msgr2=0x7f0c84100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 shutdown_connections 2024-05-11T12:25:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:06.052+0000 7f0c8b26d700 1 -- 172.21.15.47:0/1374744578 wait complete. 2024-05-11T12:25:06.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:25:06.858 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:25:07.134 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:25:07.263 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:07 smithi047 ceph-mon[30375]: pgmap v415: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:07.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:07 smithi064 ceph-mon[34537]: pgmap v415: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:07.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:07 smithi050 ceph-mon[34072]: pgmap v415: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:08.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.998+0000 7fce9ae20700 1 -- 172.21.15.47:0/2513970962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 msgr2=0x7fce941028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.998+0000 7fce9ae20700 1 --2- 172.21.15.47:0/2513970962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce941028d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fce80009990 tx=0x7fce80009ca0 comp rx=0 tx=0).stop 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.998+0000 7fce9ae20700 1 -- 172.21.15.47:0/2513970962 shutdown_connections 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.998+0000 7fce9ae20700 1 --2- 172.21.15.47:0/2513970962 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce94102e10 0x7fce9410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.998+0000 7fce9ae20700 1 --2- 172.21.15.47:0/2513970962 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce94108530 0x7fce94108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.998+0000 7fce9ae20700 1 --2- 172.21.15.47:0/2513970962 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce941028d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.998+0000 7fce9ae20700 1 -- 172.21.15.47:0/2513970962 >> 172.21.15.47:0/2513970962 conn(0x7fce94079400 msgr2=0x7fce94079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.999+0000 7fce9ae20700 1 -- 172.21.15.47:0/2513970962 shutdown_connections 2024-05-11T12:25:08.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.999+0000 7fce9ae20700 1 -- 172.21.15.47:0/2513970962 wait complete. 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.999+0000 7fce9ae20700 1 Processor -- start 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:08.999+0000 7fce9ae20700 1 -- start start 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce9ae20700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce9419bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce9ae20700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce94102e10 0x7fce9419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce9ae20700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce94108530 0x7fce941a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce9ae20700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fce941125e0 con 0x7fce94102450 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce9ae20700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fce94112460 con 0x7fce94102e10 2024-05-11T12:25:09.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce9ae20700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fce94112760 con 0x7fce94108530 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce9419bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce9419bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48796/0 (socket says 172.21.15.47:48796) 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 -- 172.21.15.47:0/375625841 learned_addr learned my addr 172.21.15.47:0/375625841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce93fff700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce94102e10 0x7fce9419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce993bd700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce94108530 0x7fce941a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 -- 172.21.15.47:0/375625841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce94108530 msgr2=0x7fce941a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce94108530 0x7fce941a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 -- 172.21.15.47:0/375625841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce94102e10 msgr2=0x7fce9419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce94102e10 0x7fce9419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 -- 172.21.15.47:0/375625841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce941a0da0 con 0x7fce94102450 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce93fff700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce94102e10 0x7fce9419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:25:09.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.000+0000 7fce98bbc700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce9419bd90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fce8800d950 tx=0x7fce8800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:09.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.001+0000 7fce91ffb700 1 -- 172.21.15.47:0/375625841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce88016070 con 0x7fce94102450 2024-05-11T12:25:09.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.001+0000 7fce91ffb700 1 -- 172.21.15.47:0/375625841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce88010460 con 0x7fce94102450 2024-05-11T12:25:09.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.001+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce941a1090 con 0x7fce94102450 2024-05-11T12:25:09.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.001+0000 7fce91ffb700 1 -- 172.21.15.47:0/375625841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8800f620 con 0x7fce94102450 2024-05-11T12:25:09.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.001+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce941a8910 con 0x7fce94102450 2024-05-11T12:25:09.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.002+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce9406a240 con 0x7fce94102450 2024-05-11T12:25:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.004+0000 7fce91ffb700 1 -- 172.21.15.47:0/375625841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fce88019070 con 0x7fce94102450 2024-05-11T12:25:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.005+0000 7fce91ffb700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce840b2880 0x7fce840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.005+0000 7fce93fff700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce840b2880 0x7fce840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.005+0000 7fce91ffb700 1 -- 172.21.15.47:0/375625841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fce880e22a0 con 0x7fce94102450 2024-05-11T12:25:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.005+0000 7fce93fff700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce840b2880 0x7fce840b4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fce80006010 tx=0x7fce8001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.009+0000 7fce91ffb700 1 -- 172.21.15.47:0/375625841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce880a9ef0 con 0x7fce94102450 2024-05-11T12:25:09.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:09 smithi047 ceph-mon[30375]: pgmap v416: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.294+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce941a8ae0 con 0x7fce94102450 2024-05-11T12:25:09.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.295+0000 7fce91ffb700 1 -- 172.21.15.47:0/375625841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fce880a97f0 con 0x7fce94102450 2024-05-11T12:25:09.295 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:25:09.295 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:25:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce840b2880 msgr2=0x7fce840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:09.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce840b2880 0x7fce840b4d40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fce80006010 tx=0x7fce8001a040 comp rx=0 tx=0).stop 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 msgr2=0x7fce9419bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce9419bd90 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fce8800d950 tx=0x7fce8800dd10 comp rx=0 tx=0).stop 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 shutdown_connections 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fce94108530 0x7fce941a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce94102450 0x7fce9419bd90 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fce840b2880 0x7fce840b4d40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 --2- 172.21.15.47:0/375625841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fce94102e10 0x7fce9419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 >> 172.21.15.47:0/375625841 conn(0x7fce94079400 msgr2=0x7fce9406b230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.297+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 shutdown_connections 2024-05-11T12:25:09.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:09.298+0000 7fce9ae20700 1 -- 172.21.15.47:0/375625841 wait complete. 2024-05-11T12:25:09.299 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:25:09.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:09 smithi064 ceph-mon[34537]: pgmap v416: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:09.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:09 smithi050 ceph-mon[34072]: pgmap v416: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:10 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/375625841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:25:10.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:10 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/375625841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:25:10.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:10 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/375625841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:25:11.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:11 smithi064 ceph-mon[34537]: pgmap v417: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:11.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:11 smithi050 ceph-mon[34072]: pgmap v417: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:11.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:11 smithi047 ceph-mon[30375]: pgmap v417: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:13.104 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:25:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:13 smithi064 ceph-mon[34537]: pgmap v418: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:13.424 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:25:13.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:13 smithi050 ceph-mon[34072]: pgmap v418: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:13 smithi047 ceph-mon[30375]: pgmap v418: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:15.164 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:15 smithi047 ceph-mon[30375]: pgmap v419: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1097334602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4102450 msgr2=0x7f6cb41028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1097334602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4102450 0x7f6cb41028d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009990 tx=0x7f6ca0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1097334602 shutdown_connections 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1097334602 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4102e10 0x7f6cb410f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1097334602 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cb4108530 0x7f6cb4108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1097334602 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4102450 0x7f6cb41028d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1097334602 >> 172.21.15.47:0/1097334602 conn(0x7f6cb4079400 msgr2=0x7f6cb4079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:15.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1097334602 shutdown_connections 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.281+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1097334602 wait complete. 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.282+0000 7f6cbb0f9700 1 Processor -- start 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.282+0000 7f6cbb0f9700 1 -- start start 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cbb0f9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cb4102450 0x7f6cb410fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cbb0f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4102e10 0x7f6cb41102d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cbb0f9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4108530 0x7f6cb4110810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cbb0f9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb4113740 con 0x7f6cb4108530 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cbb0f9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41135c0 con 0x7f6cb4102450 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cbb0f9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41138c0 con 0x7f6cb4102e10 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4108530 0x7f6cb4110810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb8e95700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cb4102450 0x7f6cb410fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4108530 0x7f6cb4110810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48808/0 (socket says 172.21.15.47:48808) 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 -- 172.21.15.47:0/1978893391 learned_addr learned my addr 172.21.15.47:0/1978893391 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4102e10 0x7f6cb41102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 -- 172.21.15.47:0/1978893391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4102e10 msgr2=0x7f6cb41102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4102e10 0x7f6cb41102d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 -- 172.21.15.47:0/1978893391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cb4102450 msgr2=0x7f6cb410fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cb4102450 0x7f6cb410fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.283+0000 7f6cb9696700 1 -- 172.21.15.47:0/1978893391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb41a8160 con 0x7f6cb4108530 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.284+0000 7f6cb9696700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4108530 0x7f6cb4110810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6cac007ee0 tx=0x7f6cac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.284+0000 7f6cb1ffb700 1 -- 172.21.15.47:0/1978893391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00dbd0 con 0x7f6cb4108530 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.284+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb41a8450 con 0x7f6cb4108530 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.284+0000 7f6cb1ffb700 1 -- 172.21.15.47:0/1978893391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cac00dd70 con 0x7f6cb4108530 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.285+0000 7f6cb1ffb700 1 -- 172.21.15.47:0/1978893391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac005990 con 0x7f6cb4108530 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.285+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb41a89e0 con 0x7f6cb4108530 2024-05-11T12:25:15.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.287+0000 7f6cb1ffb700 1 -- 172.21.15.47:0/1978893391 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6cac01a020 con 0x7f6cb4108530 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.288+0000 7f6cb1ffb700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6ca40b2880 0x7f6ca40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.288+0000 7f6cb1ffb700 1 -- 172.21.15.47:0/1978893391 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f6cac0e1840 con 0x7f6cb4108530 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.288+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6ca40b2880 0x7f6ca40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.289+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6ca40b2880 0x7f6ca40b4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005950 tx=0x7f6ca80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.290+0000 7f6cc598fb80 1 Processor -- start 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.290+0000 7f6cc598fb80 1 -- start start 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb588eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb583520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb583a60 0x555beb58f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb583520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb583520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48818/0 (socket says 172.21.15.47:48818) 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 -- 172.21.15.47:0/1407211535 learned_addr learned my addr 172.21.15.47:0/1407211535 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:15.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555beb4d5860 con 0x555beb5830a0 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555beb4747c0 con 0x555beb588ad0 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555beb4d1820 con 0x555beb583a60 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb588eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 -- 172.21.15.47:0/1407211535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb583a60 msgr2=0x555beb58f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb583a60 0x555beb58f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 -- 172.21.15.47:0/1407211535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 msgr2=0x555beb588eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb588eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 -- 172.21.15.47:0/1407211535 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555beb4228d0 con 0x555beb5830a0 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb588eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.291+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb583520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0015000 tx=0x7f6ca0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329f9698cc328a74 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.292+0000 7f6c9effd700 1 -- 172.21.15.47:0/1407211535 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0003a60 con 0x555beb5830a0 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.292+0000 7f6c9effd700 1 -- 172.21.15.47:0/1407211535 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca0026070 con 0x555beb5830a0 2024-05-11T12:25:15.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.292+0000 7f6c9effd700 1 -- 172.21.15.47:0/1407211535 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000bac0 con 0x555beb5830a0 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 msgr2=0x555beb583520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb583520 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0015000 tx=0x7f6ca0005bc0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 shutdown_connections 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb583a60 0x555beb58f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb588eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/1407211535 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb583520 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 >> 172.21.15.47:0/1407211535 conn(0x555beb4afe60 msgr2=0x555beb57d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 shutdown_connections 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 -- 172.21.15.47:0/1407211535 wait complete. 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.293+0000 7f6cc598fb80 1 -- start start 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cc598fb80 1 Processor -- start 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb5830a0 0x555beb58f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb583a60 0x555beb58c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb583a60 0x555beb58c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb583a60 0x555beb58c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48830/0 (socket says 172.21.15.47:48830) 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cb3fff700 1 -- 172.21.15.47:0/23633110 learned_addr learned my addr 172.21.15.47:0/23633110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb58ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555beb58fe30 con 0x555beb583a60 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x555beb43bc30 con 0x555beb588ad0 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.294+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x555beb58ffb0 con 0x555beb5830a0 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cc598fb80 10 client.0 _ll_register_callbacks cb 0x555beb46d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb5830a0 0x555beb58f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cb3fff700 1 -- 172.21.15.47:0/23633110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb5830a0 msgr2=0x555beb58f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb5830a0 0x555beb58f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cb3fff700 1 -- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 msgr2=0x555beb58ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb58ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cb3fff700 1 -- 172.21.15.47:0/23633110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555beb471690 con 0x555beb583a60 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca0026070 con 0x555beb583a60 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb583a60 0x555beb58c910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ba00 tx=0x7f6ca000bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9e6b3ea0aeaa6f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:15.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6c93fff700 10 client.15243 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6c93fff700 1 -- 172.21.15.47:0/23633110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0005270 con 0x555beb583a60 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6c93fff700 1 -- 172.21.15.47:0/23633110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f6ca0005410 con 0x555beb583a60 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.295+0000 7f6c93fff700 1 -- 172.21.15.47:0/23633110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6ca0005630 con 0x555beb583a60 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 10 client.15243 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=20}) -- 0x555beb59a380 con 0x555beb583a60 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6c93fff700 1 -- 172.21.15.47:0/23633110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f6ca000f460 con 0x555beb583a60 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 10 client.15243 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 10 client.15243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 20 client.15243 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb59c850 0x555beb59ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 4 client.15243 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:25:15.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555beb4e3240 con 0x555beb59c850 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.296+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb59c850 0x555beb59ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.297+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb59c850 0x555beb59ec40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bffe4d28a103ab8 server_cookie=5d2d65d5aa3d5809 in_seq=0 out_seq=0 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.297+0000 7f6c93fff700 10 client.15243 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.297+0000 7f6c93fff700 1 -- 172.21.15.47:0/23633110 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555beb4e3240 con 0x555beb59c850 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.297+0000 7f6c93fff700 10 client.15243 handle_command_reply: tid=0 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 client.15243 shutdown 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb583a60 msgr2=0x555beb58c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb583a60 0x555beb58c910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000ba00 tx=0x7f6ca000bae0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 shutdown_connections 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb59c850 0x555beb59ec40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb588ad0 0x555beb58ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb5830a0 0x555beb58f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/23633110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb583a60 0x555beb58c910 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 >> 172.21.15.47:0/23633110 conn(0x555beb4afe60 msgr2=0x555beb578460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6c93fff700 0 client.15243 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 shutdown_connections 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 1 -- 172.21.15.47:0/23633110 wait complete. 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.298+0000 7f6cc598fb80 20 client.15243 trim_cache size 0 max 16384 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 Processor -- start 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 -- start start 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb588350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb58cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb577bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41ae0a0 con 0x555beb5830a0 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41b0920 con 0x555beb58b910 2024-05-11T12:25:15.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cc598fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41ae710 con 0x555beb588ad0 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cb8e95700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb588350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb58cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cb3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb58cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:46124/0 (socket says 172.21.15.47:46124) 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.314+0000 7f6cb3fff700 1 -- 172.21.15.47:0/2496210788 learned_addr learned my addr 172.21.15.47:0/2496210788 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6cb8e95700 1 -- 172.21.15.47:0/2496210788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 msgr2=0x555beb58cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb58cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6cb8e95700 1 -- 172.21.15.47:0/2496210788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 msgr2=0x555beb577bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb577bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6cb8e95700 1 -- 172.21.15.47:0/2496210788 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555beb58ef20 con 0x555beb5830a0 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb58cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb588350 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x555beb58ffb0 tx=0x7f6ca8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd9ecbef924f101 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6c9e7fc700 1 -- 172.21.15.47:0/2496210788 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca802d070 con 0x555beb5830a0 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6c9e7fc700 1 -- 172.21.15.47:0/2496210788 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca8011d00 con 0x555beb5830a0 2024-05-11T12:25:15.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.315+0000 7f6c9e7fc700 1 -- 172.21.15.47:0/2496210788 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca801f960 con 0x555beb5830a0 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2496210788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 msgr2=0x555beb588350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb588350 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x555beb58ffb0 tx=0x7f6ca8015000 comp rx=0 tx=0).stop 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2496210788 shutdown_connections 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb577bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb588350 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2496210788 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb58cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2496210788 >> 172.21.15.47:0/2496210788 conn(0x555beb4afe60 msgr2=0x555beb58b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:15.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2496210788 shutdown_connections 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.316+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2496210788 wait complete. 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 -- start start 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 Processor -- start 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb59da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb59df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb59e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cb8e95700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb59da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41aede0 con 0x555beb5830a0 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cb8e95700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb59da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48842/0 (socket says 172.21.15.47:48842) 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41b0250 con 0x555beb58b910 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cb8e95700 1 -- 172.21.15.47:0/2624755743 learned_addr learned my addr 172.21.15.47:0/2624755743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.317+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb41af4b0 con 0x555beb588ad0 2024-05-11T12:25:15.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb59df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb9696700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb59e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cc598fb80 10 client.0 _ll_register_callbacks cb 0x555beb46d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb8e95700 1 -- 172.21.15.47:0/2624755743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 msgr2=0x555beb59df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb59df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb8e95700 1 -- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 msgr2=0x555beb59e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb59e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb8e95700 1 -- 172.21.15.47:0/2624755743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555beb616f80 con 0x555beb5830a0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb9696700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb59e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca8011d00 con 0x555beb5830a0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6cb8e95700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb59da40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6ca802b040 tx=0x7f6ca8012f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0578d1d86884939 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6c93fff700 10 client.15249 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6c93fff700 1 -- 172.21.15.47:0/2624755743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca802d040 con 0x555beb5830a0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.318+0000 7f6c93fff700 1 -- 172.21.15.47:0/2624755743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f6ca8005240 con 0x555beb5830a0 2024-05-11T12:25:15.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6c93fff700 1 -- 172.21.15.47:0/2624755743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6ca80054a0 con 0x555beb5830a0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 10 client.15249 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=20}) -- 0x555beb620950 con 0x555beb5830a0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6c93fff700 1 -- 172.21.15.47:0/2624755743 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f6ca8030030 con 0x555beb5830a0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 10 client.15249 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 10 client.15249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 20 client.15249 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb620da0 0x555beb623190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 4 client.15249 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555beb4dd2e0 con 0x555beb620da0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.319+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb620da0 0x555beb623190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6c93fff700 10 client.15249 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cb3fff700 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb620da0 0x555beb623190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e858ace40a1e665f server_cookie=cfb8352e4e799080 in_seq=0 out_seq=0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6c93fff700 1 -- 172.21.15.47:0/2624755743 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555beb4dd2e0 con 0x555beb620da0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6c93fff700 10 client.15249 handle_command_reply: tid=0 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 client.15249 shutdown 2024-05-11T12:25:15.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 msgr2=0x555beb59da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb59da40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6ca802b040 tx=0x7f6ca8012f50 comp rx=0 tx=0).stop 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 shutdown_connections 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x555beb620da0 0x555beb623190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x555beb58b910 0x555beb59e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555beb5830a0 0x555beb59da40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 --2- 172.21.15.47:0/2624755743 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x555beb588ad0 0x555beb59df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.320+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 >> 172.21.15.47:0/2624755743 conn(0x555beb4afe60 msgr2=0x555beb582090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.321+0000 7f6c93fff700 0 client.15249 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.321+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 shutdown_connections 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.321+0000 7f6cc598fb80 1 -- 172.21.15.47:0/2624755743 wait complete. 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.321+0000 7f6cc598fb80 20 client.15249 trim_cache size 0 max 16384 2024-05-11T12:25:15.333 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:25:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6ca40b2880 msgr2=0x7f6ca40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6ca40b2880 0x7f6ca40b4d40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8005950 tx=0x7f6ca80058e0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4108530 msgr2=0x7f6cb4110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4108530 0x7f6cb4110810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6cac007ee0 tx=0x7f6cac00d3b0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 shutdown_connections 2024-05-11T12:25:15.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6cb4108530 0x7f6cb4110810 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f6ca40b2880 0x7f6ca40b4d40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6cb4102450 0x7f6cb410fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 --2- 172.21.15.47:0/1978893391 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f6cb4102e10 0x7f6cb41102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 >> 172.21.15.47:0/1978893391 conn(0x7f6cb4079400 msgr2=0x7f6cb4100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 shutdown_connections 2024-05-11T12:25:15.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:15.322+0000 7f6cbb0f9700 1 -- 172.21.15.47:0/1978893391 wait complete. 2024-05-11T12:25:15.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:15 smithi064 ceph-mon[34537]: pgmap v419: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:15.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:15 smithi050 ceph-mon[34072]: pgmap v419: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:17.001 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:25:17.002 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:25:17.299 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:25:17.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:17 smithi064 ceph-mon[34537]: pgmap v420: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:17.423 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:17 smithi047 ceph-mon[30375]: pgmap v420: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:17.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:17 smithi050 ceph-mon[34072]: pgmap v420: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 -- 172.21.15.47:0/554666814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00108510 msgr2=0x7f2e001088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 --2- 172.21.15.47:0/554666814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00108510 0x7f2e001088f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009990 tx=0x7f2df0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 -- 172.21.15.47:0/554666814 shutdown_connections 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 --2- 172.21.15.47:0/554666814 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e00103fa0 0x7f2e00110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 --2- 172.21.15.47:0/554666814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00108510 0x7f2e001088f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 --2- 172.21.15.47:0/554666814 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e00103600 0x7f2e00103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 -- 172.21.15.47:0/554666814 >> 172.21.15.47:0/554666814 conn(0x7f2e0007a6b0 msgr2=0x7f2e0007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 -- 172.21.15.47:0/554666814 shutdown_connections 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.142+0000 7f2e062a3700 1 -- 172.21.15.47:0/554666814 wait complete. 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.143+0000 7f2e062a3700 1 Processor -- start 2024-05-11T12:25:19.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.143+0000 7f2e062a3700 1 -- start start 2024-05-11T12:25:19.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.143+0000 7f2e062a3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e00103600 0x7f2e00199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.143+0000 7f2e062a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00103fa0 0x7f2e0019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.143+0000 7f2e062a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e0019e6f0 0x7f2e0019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2e062a3700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001137b0 con 0x7f2e00103fa0 2024-05-11T12:25:19.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2e062a3700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00113630 con 0x7f2e00103600 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00103fa0 0x7f2e0019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00103fa0 0x7f2e0019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51286/0 (socket says 172.21.15.47:51286) 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dff7fe700 1 -- 172.21.15.47:0/2560126761 learned_addr learned my addr 172.21.15.47:0/2560126761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dfffff700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e00103600 0x7f2e00199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00113930 con 0x7f2e0019e6f0 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2e04840700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e0019e6f0 0x7f2e0019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dfffff700 1 -- 172.21.15.47:0/2560126761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e0019e6f0 msgr2=0x7f2e0019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dfffff700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e0019e6f0 0x7f2e0019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dfffff700 1 -- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00103fa0 msgr2=0x7f2e0019a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dfffff700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00103fa0 0x7f2e0019a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dfffff700 1 -- 172.21.15.47:0/2560126761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0019f210 con 0x7f2e00103600 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00103fa0 0x7f2e0019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.144+0000 7f2dfffff700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e00103600 0x7f2e00199e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e001046c0 tx=0x7f2df00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.145+0000 7f2dfd7fa700 1 -- 172.21.15.47:0/2560126761 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0026070 con 0x7f2e00103600 2024-05-11T12:25:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.145+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0019f4a0 con 0x7f2e00103600 2024-05-11T12:25:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.145+0000 7f2dfd7fa700 1 -- 172.21.15.47:0/2560126761 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df0003680 con 0x7f2e00103600 2024-05-11T12:25:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.145+0000 7f2dfd7fa700 1 -- 172.21.15.47:0/2560126761 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df000fa50 con 0x7f2e00103600 2024-05-11T12:25:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.145+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e001a6c20 con 0x7f2e00103600 2024-05-11T12:25:19.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.148+0000 7f2dfd7fa700 1 -- 172.21.15.47:0/2560126761 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2df0029070 con 0x7f2e00103600 2024-05-11T12:25:19.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 Processor -- start 2024-05-11T12:25:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 -- start start 2024-05-11T12:25:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2ef1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5561f2e429c0 con 0x5561f2ef0bb0 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561f2da7e20 con 0x5561f2ef4e00 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.149+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561f2e3e980 con 0x5561f2ef5720 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2e04840700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2ef1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dfd7fa700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2de80b2880 0x7f2de80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2e04840700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:40278/0 (socket says 172.21.15.47:40278) 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2e04840700 1 -- 172.21.15.47:0/3378976454 learned_addr learned my addr 172.21.15.47:0/3378976454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dfffff700 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dff7fe700 1 -- 172.21.15.47:0/3378976454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 msgr2=0x5561f2ef0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dff7fe700 1 -- 172.21.15.47:0/3378976454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 msgr2=0x5561f2ef51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dff7fe700 1 -- 172.21.15.47:0/3378976454 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f2d8f950 con 0x5561f2ef0bb0 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dfffff700 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.150+0000 7f2dfd7fa700 1 -- 172.21.15.47:0/2560126761 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2df00e28a0 con 0x7f2e00103600 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.151+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2ef1010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2df4016cb0 tx=0x7f2df400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ae78ae5431a0f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.151+0000 7f2de67fc700 1 -- 172.21.15.47:0/3378976454 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4024070 con 0x5561f2ef0bb0 2024-05-11T12:25:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.151+0000 7f2de67fc700 1 -- 172.21.15.47:0/3378976454 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df401f410 con 0x5561f2ef0bb0 2024-05-11T12:25:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.151+0000 7f2de67fc700 1 -- 172.21.15.47:0/3378976454 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df401e5d0 con 0x5561f2ef0bb0 2024-05-11T12:25:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.151+0000 7f2e04840700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2de80b2880 0x7f2de80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 -- 172.21.15.47:0/3378976454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 msgr2=0x5561f2ef1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e04840700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2de80b2880 0x7f2de80b4d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2df8011770 tx=0x7f2df8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2ef1010 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2df4016cb0 tx=0x7f2df400bf80 comp rx=0 tx=0).stop 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 -- 172.21.15.47:0/3378976454 shutdown_connections 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2ef1010 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/3378976454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 -- 172.21.15.47:0/3378976454 >> 172.21.15.47:0/3378976454 conn(0x5561f2e1cbf0 msgr2=0x5561f2eeb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 -- 172.21.15.47:0/3378976454 shutdown_connections 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 -- 172.21.15.47:0/3378976454 wait complete. 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.152+0000 7f2e10b39b80 1 -- start start 2024-05-11T12:25:19.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2e10b39b80 1 Processor -- start 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2efc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5561f2efd2f0 con 0x5561f2ef0bb0 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561f2dfbf50 con 0x5561f2ef4e00 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561f2efd470 con 0x5561f2ef5720 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.153+0000 7f2dfffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2efc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2efc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51308/0 (socket says 172.21.15.47:51308) 2024-05-11T12:25:19.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 -- 172.21.15.47:0/1034537941 learned_addr learned my addr 172.21.15.47:0/1034537941 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2e10b39b80 10 client.0 _ll_register_callbacks cb 0x5561f2dda9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2e04840700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 -- 172.21.15.47:0/1034537941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 msgr2=0x5561f2ef9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 -- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 msgr2=0x5561f2ef9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 -- 172.21.15.47:0/1034537941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f2df97e0 con 0x5561f2ef0bb0 2024-05-11T12:25:19.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2efc5a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2df00a7090 tx=0x7f2df002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b974ad465067f54d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2ddf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1034537941 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df00a5ce0 con 0x5561f2ef0bb0 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1034537941 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2df00b0600 con 0x5561f2ef0bb0 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.154+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df401f410 con 0x5561f2ef0bb0 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1034537941 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2df00a9590 con 0x5561f2ef0bb0 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 10 client.15261 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:19.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5561f2f07b60 con 0x5561f2ef0bb0 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1034537941 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df401f410 con 0x5561f2ef0bb0 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1034537941 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2df00a5790 con 0x5561f2ef0bb0 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 10 client.15261 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 10 client.15261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 20 client.15261 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f0c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 4 client.15261 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.155+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f2e503a0 con 0x5561f2f0a030 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.156+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f0c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.156+0000 7f2ddf7fe700 10 client.15261 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:19.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.156+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f0c4b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f5b18b5ce00e07 server_cookie=4ba86c688e2a23e9 in_seq=0 out_seq=0 2024-05-11T12:25:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1034537941 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561f2e503a0 con 0x5561f2f0a030 2024-05-11T12:25:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2ddf7fe700 10 client.15261 handle_command_reply: tid=0 2024-05-11T12:25:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 client.15261 shutdown 2024-05-11T12:25:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 msgr2=0x5561f2efc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2efc5a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2df00a7090 tx=0x7f2df002e000 comp rx=0 tx=0).stop 2024-05-11T12:25:19.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 shutdown_connections 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f0c4b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ef9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef0bb0 0x5561f2efc5a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1034537941 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2ef9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2ddf7fe700 0 client.15261 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.157+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 >> 172.21.15.47:0/1034537941 conn(0x5561f2e1cbf0 msgr2=0x5561f2ee58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.158+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 shutdown_connections 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.158+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1034537941 wait complete. 2024-05-11T12:25:19.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.158+0000 7f2e10b39b80 20 client.15261 trim_cache size 0 max 16384 2024-05-11T12:25:19.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 Processor -- start 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 -- start start 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef0bb0 0x5561f2f06380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef4e00 0x5561f2ef8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ee59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001aa270 con 0x5561f2ef4e00 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001acaf0 con 0x5561f2ef0bb0 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001aa8e0 con 0x5561f2ef5720 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2dff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef4e00 0x5561f2ef8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2dff7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef4e00 0x5561f2ef8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51316/0 (socket says 172.21.15.47:51316) 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2dff7fe700 1 -- 172.21.15.47:0/1452210170 learned_addr learned my addr 172.21.15.47:0/1452210170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef0bb0 0x5561f2f06380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.192+0000 7f2e04840700 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ee59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2dff7fe700 1 -- 172.21.15.47:0/1452210170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 msgr2=0x5561f2ee59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ee59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2dff7fe700 1 -- 172.21.15.47:0/1452210170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef0bb0 msgr2=0x5561f2f06380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef0bb0 0x5561f2f06380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2dff7fe700 1 -- 172.21.15.47:0/1452210170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f2eec150 con 0x5561f2ef4e00 2024-05-11T12:25:19.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef4e00 0x5561f2ef8af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x5561f2efaf10 tx=0x7f2df400b280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d63d86276cc58a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:19.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2de5ffb700 1 -- 172.21.15.47:0/1452210170 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4030020 con 0x5561f2ef4e00 2024-05-11T12:25:19.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2de5ffb700 1 -- 172.21.15.47:0/1452210170 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df4025040 con 0x5561f2ef4e00 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.193+0000 7f2de5ffb700 1 -- 172.21.15.47:0/1452210170 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4024070 con 0x5561f2ef4e00 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1452210170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef4e00 msgr2=0x5561f2ef8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef4e00 0x5561f2ef8af0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x5561f2efaf10 tx=0x7f2df400b280 comp rx=0 tx=0).stop 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1452210170 shutdown_connections 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef5720 0x5561f2ee59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef0bb0 0x5561f2f06380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1452210170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef4e00 0x5561f2ef8af0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1452210170 >> 172.21.15.47:0/1452210170 conn(0x5561f2e1cbf0 msgr2=0x5561f2efb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1452210170 shutdown_connections 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.194+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1452210170 wait complete. 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 -- start start 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 Processor -- start 2024-05-11T12:25:19.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef0bb0 0x5561f2f02700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2f02c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef5720 0x5561f2eed570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001aafb0 con 0x5561f2ef5720 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001ac420 con 0x5561f2ef4e00 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001ab680 con 0x5561f2ef0bb0 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e10b39b80 10 client.0 _ll_register_callbacks cb 0x5561f2dda9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.195+0000 7f2e04840700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef5720 0x5561f2eed570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef5720 0x5561f2eed570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51326/0 (socket says 172.21.15.47:51326) 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 -- 172.21.15.47:0/1326825974 learned_addr learned my addr 172.21.15.47:0/1326825974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2f02c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef0bb0 0x5561f2f02700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 -- 172.21.15.47:0/1326825974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef0bb0 msgr2=0x5561f2f02700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef0bb0 0x5561f2f02700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 -- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 msgr2=0x5561f2f02c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2f02c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 -- 172.21.15.47:0/1326825974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f2eedab0 con 0x5561f2ef5720 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef0bb0 0x5561f2f02700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2dff7fe700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2f02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2e04840700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef5720 0x5561f2eed570 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2df8020d90 tx=0x7f2df8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25413d91275da86d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2ddf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1326825974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8017070 con 0x5561f2ef5720 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.196+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1326825974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2df801cdc0 con 0x5561f2ef5720 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df4025040 con 0x5561f2ef5720 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1326825974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2df801e440 con 0x5561f2ef5720 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 10 client.15267 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:19.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5561f2f76100 con 0x5561f2ef5720 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1326825974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2df4025040 con 0x5561f2ef5720 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1326825974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2df8034700 con 0x5561f2ef5720 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 10 client.15267 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 10 client.15267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 20 client.15267 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 4 client.15267 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:25:19.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.197+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5561f2eeb5c0 con 0x5561f2f0a030 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.198+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.198+0000 7f2ddf7fe700 10 client.15267 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.198+0000 7f2dfffff700 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f788b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd13478a453200cb server_cookie=22e23af2d98fc8ad in_seq=0 out_seq=0 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.199+0000 7f2ddf7fe700 1 -- 172.21.15.47:0/1326825974 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5561f2eeb5c0 con 0x5561f2f0a030 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.199+0000 7f2ddf7fe700 10 client.15267 handle_command_reply: tid=0 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.199+0000 7f2e10b39b80 1 client.15267 shutdown 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef5720 msgr2=0x5561f2eed570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef5720 0x5561f2eed570 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2df8020d90 tx=0x7f2df8004ad0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 shutdown_connections 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561f2f0a030 0x5561f2f788b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2ddf7fe700 0 client.15267 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561f2ef5720 0x5561f2eed570 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561f2ef0bb0 0x5561f2f02700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 --2- 172.21.15.47:0/1326825974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561f2ef4e00 0x5561f2f02c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 >> 172.21.15.47:0/1326825974 conn(0x5561f2e1cbf0 msgr2=0x5561f2efaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 shutdown_connections 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 1 -- 172.21.15.47:0/1326825974 wait complete. 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.200+0000 7f2e10b39b80 20 client.15267 trim_cache size 0 max 16384 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "d7295d29-c36f-42e1-bb51-2d657bde43b7", 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:25:19.203 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2de80b2880 msgr2=0x7f2de80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2de80b2880 0x7f2de80b4d40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2df8011770 tx=0x7f2df8009040 comp rx=0 tx=0).stop 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e00103600 msgr2=0x7f2e00199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e00103600 0x7f2e00199e20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2e001046c0 tx=0x7f2df00047b0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 shutdown_connections 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2e0019e6f0 0x7f2e0019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2de80b2880 0x7f2de80b4d40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2e00103600 0x7f2e00199e20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 --2- 172.21.15.47:0/2560126761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2e00103fa0 0x7f2e0019a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 >> 172.21.15.47:0/2560126761 conn(0x7f2e0007a6b0 msgr2=0x7f2e001017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:19.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 shutdown_connections 2024-05-11T12:25:19.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:19.201+0000 7f2e062a3700 1 -- 172.21.15.47:0/2560126761 wait complete. 2024-05-11T12:25:19.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:19 smithi047 ceph-mon[30375]: pgmap v421: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:19.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:19 smithi064 ceph-mon[34537]: pgmap v421: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:19 smithi050 ceph-mon[34072]: pgmap v421: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:20.017 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:25:20.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:20 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:25:20.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:20 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:25:20.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:20 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:25:20.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:20 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:25:20.278 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:25:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:20 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:25:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:20 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:25:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:20 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:25:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:20 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:25:20.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:20 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:25:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:20 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:25:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:20 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:25:20.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:20 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:25:21.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:21 smithi064 ceph-mon[34537]: pgmap v422: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:21.433 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:21 smithi047 ceph-mon[30375]: pgmap v422: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:21.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:21 smithi050 ceph-mon[34072]: pgmap v422: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 -- 172.21.15.47:0/392955050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c108530 msgr2=0x7f118c108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 --2- 172.21.15.47:0/392955050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c108530 0x7f118c108910 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f117c009990 tx=0x7f117c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 -- 172.21.15.47:0/392955050 shutdown_connections 2024-05-11T12:25:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 --2- 172.21.15.47:0/392955050 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f118c102e10 0x7f118c10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 --2- 172.21.15.47:0/392955050 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c108530 0x7f118c108910 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 --2- 172.21.15.47:0/392955050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c102450 0x7f118c1028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 -- 172.21.15.47:0/392955050 >> 172.21.15.47:0/392955050 conn(0x7f118c079400 msgr2=0x7f118c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 -- 172.21.15.47:0/392955050 shutdown_connections 2024-05-11T12:25:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.145+0000 7f11918a1700 1 -- 172.21.15.47:0/392955050 wait complete. 2024-05-11T12:25:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.147+0000 7f11918a1700 1 Processor -- start 2024-05-11T12:25:22.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.147+0000 7f11918a1700 1 -- start start 2024-05-11T12:25:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.147+0000 7f11918a1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f118c102450 0x7f118c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.147+0000 7f11918a1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c102e10 0x7f118c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f11918a1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c19d640 0x7f118c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f11918a1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f118c112630 con 0x7f118c102e10 2024-05-11T12:25:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f11918a1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1124b0 con 0x7f118c19d640 2024-05-11T12:25:22.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f11918a1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1127b0 con 0x7f118c102450 2024-05-11T12:25:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c102e10 0x7f118c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c102e10 0x7f118c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51344/0 (socket says 172.21.15.47:51344) 2024-05-11T12:25:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 -- 172.21.15.47:0/827616109 learned_addr learned my addr 172.21.15.47:0/827616109 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:22.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118affd700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f118c102450 0x7f118c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118b7fe700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c19d640 0x7f118c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 -- 172.21.15.47:0/827616109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f118c102450 msgr2=0x7f118c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f118c102450 0x7f118c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 -- 172.21.15.47:0/827616109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c19d640 msgr2=0x7f118c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c19d640 0x7f118c19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.148+0000 7f118a7fc700 1 -- 172.21.15.47:0/827616109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c19e160 con 0x7f118c102e10 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.149+0000 7f118a7fc700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c102e10 0x7f118c1992b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9a0 tx=0x7f118000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.149+0000 7f119089f700 1 -- 172.21.15.47:0/827616109 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118001b070 con 0x7f118c102e10 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.149+0000 7f119089f700 1 -- 172.21.15.47:0/827616109 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f118000de90 con 0x7f118c102e10 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.149+0000 7f119089f700 1 -- 172.21.15.47:0/827616109 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800153b0 con 0x7f118c102e10 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.149+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118c19e450 con 0x7f118c102e10 2024-05-11T12:25:22.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.149+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f118c06b520 con 0x7f118c102e10 2024-05-11T12:25:22.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.153+0000 7f119089f700 1 -- 172.21.15.47:0/827616109 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f118001e070 con 0x7f118c102e10 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.154+0000 7f119089f700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11740b2900 0x7f11740b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.154+0000 7f119c137b80 1 Processor -- start 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.154+0000 7f118affd700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11740b2900 0x7f11740b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.154+0000 7f119089f700 1 -- 172.21.15.47:0/827616109 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f11800e1850 con 0x7f118c102e10 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.154+0000 7f119c137b80 1 -- start start 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.154+0000 7f119c137b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c7bad0 0x563ed8c7beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f119c137b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c76520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f119c137b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f119c137b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563ed8bc8860 con 0x563ed8c760a0 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f119c137b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ed8b677c0 con 0x563ed8c7bad0 2024-05-11T12:25:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f119c137b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563ed8bc4820 con 0x563ed8c76a60 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c76520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118affd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c7bad0 0x563ed8c7beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c76520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51350/0 (socket says 172.21.15.47:51350) 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 -- 172.21.15.47:0/2520623068 learned_addr learned my addr 172.21.15.47:0/2520623068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118b7fe700 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118affd700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11740b2900 0x7f11740b4dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098e0 tx=0x7f117c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 -- 172.21.15.47:0/2520623068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 msgr2=0x563ed8c82890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c82890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 -- 172.21.15.47:0/2520623068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c7bad0 msgr2=0x563ed8c7beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c7bad0 0x563ed8c7beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118a7fc700 1 -- 172.21.15.47:0/2520623068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed8b158d0 con 0x563ed8c760a0 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.155+0000 7f118b7fe700 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c82890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.156+0000 7f118a7fc700 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c76520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f11800a8a40 tx=0x7f1180017fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb54683c37c098de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:22.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.156+0000 7f1171ffb700 1 -- 172.21.15.47:0/2520623068 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800a8d40 con 0x563ed8c760a0 2024-05-11T12:25:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.156+0000 7f1171ffb700 1 -- 172.21.15.47:0/2520623068 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11800ad0f0 con 0x563ed8c760a0 2024-05-11T12:25:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.156+0000 7f1171ffb700 1 -- 172.21.15.47:0/2520623068 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11800b1870 con 0x563ed8c760a0 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 -- 172.21.15.47:0/2520623068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 msgr2=0x563ed8c76520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c76520 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f11800a8a40 tx=0x7f1180017fa0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 -- 172.21.15.47:0/2520623068 shutdown_connections 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c82890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c7bad0 0x563ed8c7beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 --2- 172.21.15.47:0/2520623068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c76520 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 -- 172.21.15.47:0/2520623068 >> 172.21.15.47:0/2520623068 conn(0x563ed8ba2e60 msgr2=0x563ed8c70990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 -- 172.21.15.47:0/2520623068 shutdown_connections 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.157+0000 7f119c137b80 1 -- 172.21.15.47:0/2520623068 wait complete. 2024-05-11T12:25:22.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 -- start start 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 Processor -- start 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c760a0 0x563ed8c82550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 0x563ed8c7f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c7fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c7fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f118b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c7fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51356/0 (socket says 172.21.15.47:51356) 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f118b7fe700 1 -- 172.21.15.47:0/1921442128 learned_addr learned my addr 172.21.15.47:0/1921442128 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563ed8c82e30 con 0x563ed8c7bad0 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563ed8b2ec30 con 0x563ed8c76a60 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.158+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563ed8c82fb0 con 0x563ed8c760a0 2024-05-11T12:25:22.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118a7fc700 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 0x563ed8c7f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118affd700 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c760a0 0x563ed8c82550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f119c137b80 10 client.0 _ll_register_callbacks cb 0x563ed8b60920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118b7fe700 1 -- 172.21.15.47:0/1921442128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c760a0 msgr2=0x563ed8c82550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118b7fe700 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c760a0 0x563ed8c82550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118b7fe700 1 -- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 msgr2=0x563ed8c7f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118b7fe700 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 0x563ed8c7f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118b7fe700 1 -- 172.21.15.47:0/1921442128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed8b64690 con 0x563ed8c7bad0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11800ad0f0 con 0x563ed8c7bad0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f118b7fe700 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c7fe30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1184008b70 tx=0x7f1184008e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed51fbee0da1625d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f116affd700 10 client.15279 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f116affd700 1 -- 172.21.15.47:0/1921442128 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118401d030 con 0x563ed8c7bad0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.159+0000 7f116affd700 1 -- 172.21.15.47:0/1921442128 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1184004bb0 con 0x563ed8c7bad0 2024-05-11T12:25:22.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f116affd700 1 -- 172.21.15.47:0/1921442128 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1184004e10 con 0x563ed8c7bad0 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 10 client.15279 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=20}) -- 0x563ed8c8d440 con 0x563ed8c7bad0 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f116affd700 1 -- 172.21.15.47:0/1921442128 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1184012900 con 0x563ed8c7bad0 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 10 client.15279 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 10 client.15279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 20 client.15279 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8c8f970 0x563ed8c91d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 4 client.15279 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed8bd6240 con 0x563ed8c8f970 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.160+0000 7f118affd700 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8c8f970 0x563ed8c91d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.161+0000 7f118affd700 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8c8f970 0x563ed8c91d60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46eb7dac0162a710 server_cookie=9fcbcd24fae5ab26 in_seq=0 out_seq=0 2024-05-11T12:25:22.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.161+0000 7f116affd700 10 client.15279 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f116affd700 1 -- 172.21.15.47:0/1921442128 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563ed8bd6240 con 0x563ed8c8f970 2024-05-11T12:25:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f116affd700 10 client.15279 handle_command_reply: tid=0 2024-05-11T12:25:22.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 client.15279 shutdown 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 msgr2=0x563ed8c7fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c7fe30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1184008b70 tx=0x7f1184008e80 comp rx=0 tx=0).stop 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 shutdown_connections 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8c8f970 0x563ed8c91d60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c7fe30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c760a0 0x563ed8c82550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f116affd700 0 client.15279 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 --2- 172.21.15.47:0/1921442128 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 0x563ed8c7f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 >> 172.21.15.47:0/1921442128 conn(0x563ed8ba2e60 msgr2=0x563ed8c6b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 shutdown_connections 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 1 -- 172.21.15.47:0/1921442128 wait complete. 2024-05-11T12:25:22.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.162+0000 7f119c137b80 20 client.15279 trim_cache size 0 max 16384 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.202+0000 7f119c137b80 1 Processor -- start 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.202+0000 7f119c137b80 1 -- start start 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f119c137b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c8be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f119c137b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 0x563ed8c7f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f119c137b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c7bad0 0x563ed8c6abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f119c137b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1aa4c0 con 0x563ed8c760a0 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f119c137b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1acd40 con 0x563ed8c76a60 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f119c137b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1aab30 con 0x563ed8c7bad0 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c8be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c8be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51360/0 (socket says 172.21.15.47:51360) 2024-05-11T12:25:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 -- 172.21.15.47:0/1493230500 learned_addr learned my addr 172.21.15.47:0/1493230500 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118b7fe700 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c7bad0 0x563ed8c6abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 -- 172.21.15.47:0/1493230500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c7bad0 msgr2=0x563ed8c6abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c7bad0 0x563ed8c6abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 -- 172.21.15.47:0/1493230500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 msgr2=0x563ed8c7f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 0x563ed8c7f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.203+0000 7f118affd700 1 -- 172.21.15.47:0/1493230500 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed8c922f0 con 0x563ed8c760a0 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.204+0000 7f118affd700 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c8be80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b2e0 tx=0x7f117c02d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9babae19daab5c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.204+0000 7f11717fa700 1 -- 172.21.15.47:0/1493230500 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c044070 con 0x563ed8c760a0 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.204+0000 7f11717fa700 1 -- 172.21.15.47:0/1493230500 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f117c00bb00 con 0x563ed8c760a0 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.204+0000 7f11717fa700 1 -- 172.21.15.47:0/1493230500 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c02ddd0 con 0x563ed8c760a0 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 -- 172.21.15.47:0/1493230500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 msgr2=0x563ed8c8be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c8be80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f117c00b2e0 tx=0x7f117c02d000 comp rx=0 tx=0).stop 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 -- 172.21.15.47:0/1493230500 shutdown_connections 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c7bad0 0x563ed8c6abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c760a0 0x563ed8c8be80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 --2- 172.21.15.47:0/1493230500 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c76a60 0x563ed8c7f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 -- 172.21.15.47:0/1493230500 >> 172.21.15.47:0/1493230500 conn(0x563ed8ba2e60 msgr2=0x563ed8c75920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 -- 172.21.15.47:0/1493230500 shutdown_connections 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 -- 172.21.15.47:0/1493230500 wait complete. 2024-05-11T12:25:22.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.205+0000 7f119c137b80 1 -- start start 2024-05-11T12:25:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.206+0000 7f119c137b80 1 Processor -- start 2024-05-11T12:25:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.206+0000 7f119c137b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c760a0 0x563ed8c883e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.206+0000 7f119c137b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c88920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.206+0000 7f119c137b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c85250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.206+0000 7f119c137b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1ab200 con 0x563ed8c7bad0 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c88920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118a7fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c88920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:51974/0 (socket says 172.21.15.47:51974) 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118a7fc700 1 -- 172.21.15.47:0/1042551891 learned_addr learned my addr 172.21.15.47:0/1042551891 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118affd700 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c760a0 0x563ed8c883e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118b7fe700 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c85250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1ac670 con 0x563ed8c760a0 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1ab8d0 con 0x563ed8c76a60 2024-05-11T12:25:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f119c137b80 10 client.0 _ll_register_callbacks cb 0x563ed8b60920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118affd700 1 -- 172.21.15.47:0/1042551891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 msgr2=0x563ed8c88920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118affd700 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c88920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118affd700 1 -- 172.21.15.47:0/1042551891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 msgr2=0x563ed8c85250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118affd700 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c85250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118affd700 1 -- 172.21.15.47:0/1042551891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed8c85a90 con 0x563ed8c760a0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f117c00bb00 con 0x563ed8c760a0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f118affd700 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c760a0 0x563ed8c883e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f117c000c00 tx=0x7f117c00fbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10447a0f3e4147ac server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.207+0000 7f116affd700 10 client.24908 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f116affd700 1 -- 172.21.15.47:0/1042551891 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117c044070 con 0x563ed8c760a0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f116affd700 1 -- 172.21.15.47:0/1042551891 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f117c03e3d0 con 0x563ed8c760a0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f116affd700 1 -- 172.21.15.47:0/1042551891 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f117c03e5d0 con 0x563ed8c760a0 2024-05-11T12:25:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 10 client.24908 fetch_fsmap learned FSMap version 20 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({fsmap=20}) -- 0x563ed8cfab90 con 0x563ed8c760a0 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f116affd700 1 -- 172.21.15.47:0/1042551891 <== mon.2 v2:172.21.15.50:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f117c02c070 con 0x563ed8c760a0 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 10 client.24908 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 10 client.24908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 20 client.24908 populate_metadata read hostname 'smithi047' 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8cfb080 0x563ed8cfd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 4 client.24908 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.208+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ed8bd02e0 con 0x563ed8cfb080 2024-05-11T12:25:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.209+0000 7f118a7fc700 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8cfb080 0x563ed8cfd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.209+0000 7f116affd700 10 client.24908 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.209+0000 7f118a7fc700 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8cfb080 0x563ed8cfd470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d439ed0afd25217 server_cookie=3108f829c395efbd in_seq=0 out_seq=0 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.209+0000 7f116affd700 1 -- 172.21.15.47:0/1042551891 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563ed8bd02e0 con 0x563ed8cfb080 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.209+0000 7f116affd700 10 client.24908 handle_command_reply: tid=0 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.209+0000 7f119c137b80 1 client.24908 shutdown 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c760a0 msgr2=0x563ed8c883e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c760a0 0x563ed8c883e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f117c000c00 tx=0x7f117c00fbd0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 shutdown_connections 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563ed8cfb080 0x563ed8cfd470 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563ed8c7bad0 0x563ed8c85250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563ed8c760a0 0x563ed8c883e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f116affd700 0 client.24908 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 --2- 172.21.15.47:0/1042551891 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563ed8c76a60 0x563ed8c88920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 >> 172.21.15.47:0/1042551891 conn(0x563ed8ba2e60 msgr2=0x563ed8c79550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 shutdown_connections 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 1 -- 172.21.15.47:0/1042551891 wait complete. 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.210+0000 7f119c137b80 20 client.24908 trim_cache size 0 max 16384 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": { 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "d7295d29-c36f-42e1-bb51-2d657bde43b7": { 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "path": "/", 2024-05-11T12:25:22.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tag": "d7295d29-c36f-42e1-bb51-2d657bde43b7", 2024-05-11T12:25:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: "options": "recursive,force" 2024-05-11T12:25:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:25:22.215 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:25:22.215 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.211+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11740b2900 msgr2=0x7f11740b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.211+0000 7f11918a1700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11740b2900 0x7f11740b4dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098e0 tx=0x7f117c005fb0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.211+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c102e10 msgr2=0x7f118c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.211+0000 7f11918a1700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c102e10 0x7f118c1992b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f118000e9a0 tx=0x7f118000ecb0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.211+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 shutdown_connections 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.211+0000 7f11918a1700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f11740b2900 0x7f11740b4dc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.211+0000 7f11918a1700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f118c19d640 0x7f118c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.212+0000 7f11918a1700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f118c102450 0x7f118c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.212+0000 7f11918a1700 1 --2- 172.21.15.47:0/827616109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f118c102e10 0x7f118c1992b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.212+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 >> 172.21.15.47:0/827616109 conn(0x7f118c079400 msgr2=0x7f118c101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.212+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 shutdown_connections 2024-05-11T12:25:22.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:25:22.212+0000 7f11918a1700 1 -- 172.21.15.47:0/827616109 wait complete. 2024-05-11T12:25:23.076 INFO:tasks.cephfs.filesystem:scrub status for tag:d7295d29-c36f-42e1-bb51-2d657bde43b7 - {'path': '/', 'tag': 'd7295d29-c36f-42e1-bb51-2d657bde43b7', 'options': 'recursive,force'} 2024-05-11T12:25:23.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:23 smithi064 ceph-mon[34537]: pgmap v423: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:25:23.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:23 smithi050 ceph-mon[34072]: pgmap v423: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:25:23.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:23 smithi047 ceph-mon[30375]: pgmap v423: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:25:24.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:24 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:25:24.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:24 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:25:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:24 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:25:25.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:25 smithi064 ceph-mon[34537]: pgmap v424: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:25:25.407 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:25 smithi047 ceph-mon[30375]: pgmap v424: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:25:25.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:25 smithi050 ceph-mon[34072]: pgmap v424: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:25:26.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:26 smithi050 ceph-mon[34072]: pgmap v425: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:25:26.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:26 smithi047 ceph-mon[30375]: pgmap v425: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:25:26.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:26 smithi064 ceph-mon[34537]: pgmap v425: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:25:28.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:28 smithi050 ceph-mon[34072]: pgmap v426: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:25:29.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:28 smithi047 ceph-mon[30375]: pgmap v426: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:25:29.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:28 smithi064 ceph-mon[34537]: pgmap v426: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:25:29.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:25:30.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:25:30.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:25:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:30 smithi050 ceph-mon[34072]: pgmap v427: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:25:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:30 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:25:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:30 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:25:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:30 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:25:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:30 smithi047 ceph-mon[30375]: pgmap v427: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:25:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:30 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:25:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:30 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:25:31.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:30 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:25:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:30 smithi064 ceph-mon[34537]: pgmap v427: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-05-11T12:25:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:30 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:25:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:30 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:25:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:30 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:25:32.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:32 smithi050 ceph-mon[34072]: pgmap v428: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:25:33.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:32 smithi047 ceph-mon[30375]: pgmap v428: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:25:33.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:32 smithi064 ceph-mon[34537]: pgmap v428: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-05-11T12:25:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:34 smithi050 ceph-mon[34072]: pgmap v429: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:25:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:34 smithi047 ceph-mon[30375]: pgmap v429: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:25:35.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:34 smithi064 ceph-mon[34537]: pgmap v429: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-05-11T12:25:36.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:36 smithi050 ceph-mon[34072]: pgmap v430: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:25:37.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:36 smithi047 ceph-mon[30375]: pgmap v430: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:25:37.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:36 smithi064 ceph-mon[34537]: pgmap v430: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-05-11T12:25:38.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:38 smithi050 ceph-mon[34072]: pgmap v431: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:39.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:38 smithi047 ceph-mon[30375]: pgmap v431: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:39.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:38 smithi064 ceph-mon[34537]: pgmap v431: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:40.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:40 smithi050 ceph-mon[34072]: pgmap v432: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:41.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:40 smithi047 ceph-mon[30375]: pgmap v432: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:40 smithi064 ceph-mon[34537]: pgmap v432: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:42.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:42 smithi050 ceph-mon[34072]: pgmap v433: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:43.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:42 smithi047 ceph-mon[30375]: pgmap v433: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:42 smithi064 ceph-mon[34537]: pgmap v433: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:44.834 INFO:tasks.workunit.client.0.smithi047.stderr:Updating files: 92% (11343/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-11T12:25:44.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:44 smithi050 ceph-mon[34072]: pgmap v434: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:45.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:44 smithi047 ceph-mon[30375]: pgmap v434: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:44 smithi064 ceph-mon[34537]: pgmap v434: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:45.706 INFO:tasks.workunit.client.0.smithi047.stderr:Note: switching to '308506802b0bdea4df7050cfc1fcbdc04fea6a6e'. 2024-05-11T12:25:45.706 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:45.706 INFO:tasks.workunit.client.0.smithi047.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-11T12:25:45.706 INFO:tasks.workunit.client.0.smithi047.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-11T12:25:45.706 INFO:tasks.workunit.client.0.smithi047.stderr:state without impacting any branches by switching back to a branch. 2024-05-11T12:25:45.706 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:45.706 INFO:tasks.workunit.client.0.smithi047.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr: git switch -c 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr:Or undo this operation with: 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr: git switch - 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:45.707 INFO:tasks.workunit.client.0.smithi047.stderr:HEAD is now at 308506802b0 Merge PR #55956 into wip-rishabh-testing-20240510.053205 2024-05-11T12:25:45.712 DEBUG:teuthology.orchestra.run.smithi047:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-11T12:25:45.742 INFO:tasks.workunit.client.0.smithi047.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-11T12:25:45.745 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-11T12:25:45.745 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-11T12:25:46.422 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-11T12:25:46.478 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-11T12:25:46.520 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-11T12:25:46.521 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-11T12:25:46.521 INFO:tasks.workunit.client.0.smithi047.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-11T12:25:46.562 INFO:tasks.workunit.client.0.smithi047.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-11T12:25:46.566 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:25:46.566 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-11T12:25:46.624 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-05-11T12:25:46.625 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-05-11T12:25:46.625 DEBUG:teuthology.orchestra.run.smithi047:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=308506802b0bdea4df7050cfc1fcbdc04fea6a6e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-05-11T12:25:46.686 INFO:tasks.workunit.client.0.smithi047.stdout:getting iogen 2024-05-11T12:25:46.687 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'getting iogen' 2024-05-11T12:25:46.687 INFO:tasks.workunit.client.0.smithi047.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-11T12:25:46.712 INFO:tasks.workunit.client.0.smithi047.stderr:--2024-05-11 12:25:46-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-05-11T12:25:46.722 INFO:tasks.workunit.client.0.smithi047.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-05-11T12:25:46.746 INFO:tasks.workunit.client.0.smithi047.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-05-11T12:25:46.764 INFO:tasks.workunit.client.0.smithi047.stderr:HTTP request sent, awaiting response... 200 OK 2024-05-11T12:25:46.764 INFO:tasks.workunit.client.0.smithi047.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-05-11T12:25:46.766 INFO:tasks.workunit.client.0.smithi047.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-05-11T12:25:46.769 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:46.769 INFO:tasks.workunit.client.0.smithi047.stderr: 0K ....... 100% 1.31G=0s 2024-05-11T12:25:46.770 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:46.770 INFO:tasks.workunit.client.0.smithi047.stderr:2024-05-11 12:25:46 (1.31 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-05-11T12:25:46.770 INFO:tasks.workunit.client.0.smithi047.stderr: 2024-05-11T12:25:46.770 INFO:tasks.workunit.client.0.smithi047.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-05-11T12:25:46.785 INFO:tasks.workunit.client.0.smithi047.stdout:iogen_3.1p0 2024-05-11T12:25:46.786 INFO:tasks.workunit.client.0.smithi047.stdout:iogen_3.1p0/src 2024-05-11T12:25:46.787 INFO:tasks.workunit.client.0.smithi047.stdout:iogen_3.1p0/src/iogen.8 2024-05-11T12:25:46.788 INFO:tasks.workunit.client.0.smithi047.stdout:iogen_3.1p0/src/iogen.c 2024-05-11T12:25:46.789 INFO:tasks.workunit.client.0.smithi047.stdout:iogen_3.1p0/README 2024-05-11T12:25:46.790 INFO:tasks.workunit.client.0.smithi047.stdout:iogen_3.1p0/Makefile.bsd 2024-05-11T12:25:46.791 INFO:tasks.workunit.client.0.smithi047.stdout:iogen_3.1p0/Makefile 2024-05-11T12:25:46.792 INFO:tasks.workunit.client.0.smithi047.stderr:+ cd iogen_3.1p0 2024-05-11T12:25:46.792 INFO:tasks.workunit.client.0.smithi047.stdout:making iogen 2024-05-11T12:25:46.793 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'making iogen' 2024-05-11T12:25:46.793 INFO:tasks.workunit.client.0.smithi047.stderr:+ make 2024-05-11T12:25:46.801 INFO:tasks.workunit.client.0.smithi047.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-05-11T12:25:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:46 smithi050 ceph-mon[34072]: pgmap v435: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:47.024 INFO:tasks.workunit.client.0.smithi047.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-05-11T12:25:47.024 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:46 smithi047 ceph-mon[30375]: pgmap v435: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:47.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:46 smithi064 ceph-mon[34537]: pgmap v435: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-05-11T12:25:47.151 INFO:tasks.workunit.client.0.smithi047.stdout:running iogen 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'running iogen' 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./iogen -n 5 -s 2g 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:child spawned 87298 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:child spawned 87299 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:child spawned 87300 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:child spawned 87301 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:child spawned 87302 2024-05-11T12:25:47.152 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'sleep for 10 min' 2024-05-11T12:25:47.153 INFO:tasks.workunit.client.0.smithi047.stdout:sleep for 10 min 2024-05-11T12:25:47.153 INFO:tasks.workunit.client.0.smithi047.stderr:+ sleep 600 2024-05-11T12:25:48.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:48 smithi050 ceph-mon[34072]: pgmap v436: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:49.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:48 smithi047 ceph-mon[30375]: pgmap v436: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:48 smithi064 ceph-mon[34537]: pgmap v436: 97 pgs: 97 active+clean; 654 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-05-11T12:25:51.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:50 smithi047 ceph-mon[30375]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 5.3 MiB/s wr, 3 op/s 2024-05-11T12:25:51.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:50 smithi064 ceph-mon[34537]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 5.3 MiB/s wr, 3 op/s 2024-05-11T12:25:51.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:50 smithi050 ceph-mon[34072]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 5.3 MiB/s wr, 3 op/s 2024-05-11T12:25:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:52 smithi047 ceph-mon[30375]: pgmap v438: 97 pgs: 97 active+clean; 948 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 79 MiB/s wr, 93 op/s 2024-05-11T12:25:53.079 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:25:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:52 smithi064 ceph-mon[34537]: pgmap v438: 97 pgs: 97 active+clean; 948 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 79 MiB/s wr, 93 op/s 2024-05-11T12:25:53.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:52 smithi050 ceph-mon[34072]: pgmap v438: 97 pgs: 97 active+clean; 948 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 79 MiB/s wr, 93 op/s 2024-05-11T12:25:53.648 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:25:55.001 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:54 smithi050 ceph-mon[34072]: pgmap v439: 97 pgs: 97 active+clean; 3.0 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 258 MiB/s wr, 233 op/s 2024-05-11T12:25:55.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:54 smithi047 ceph-mon[30375]: pgmap v439: 97 pgs: 97 active+clean; 3.0 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 258 MiB/s wr, 233 op/s 2024-05-11T12:25:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:54 smithi064 ceph-mon[34537]: pgmap v439: 97 pgs: 97 active+clean; 3.0 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 258 MiB/s wr, 233 op/s 2024-05-11T12:25:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:56 smithi047 ceph-mon[30375]: pgmap v440: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 353 MiB/s wr, 270 op/s 2024-05-11T12:25:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:56 smithi064 ceph-mon[34537]: pgmap v440: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 353 MiB/s wr, 270 op/s 2024-05-11T12:25:57.165 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:56 smithi050 ceph-mon[34072]: pgmap v440: 97 pgs: 97 active+clean; 4.1 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 353 MiB/s wr, 270 op/s 2024-05-11T12:25:58.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:25:58 smithi047 ceph-mon[30375]: pgmap v441: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 577 MiB/s wr, 677 op/s 2024-05-11T12:25:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:25:58 smithi064 ceph-mon[34537]: pgmap v441: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 577 MiB/s wr, 677 op/s 2024-05-11T12:25:59.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:25:58 smithi050 ceph-mon[34072]: pgmap v441: 97 pgs: 97 active+clean; 6.5 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 577 MiB/s wr, 677 op/s 2024-05-11T12:26:00.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.493+0000 7fd43359e700 1 -- 172.21.15.47:0/388410444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43411cd30 msgr2=0x7fd43411f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.493+0000 7fd43359e700 1 --2- 172.21.15.47:0/388410444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43411cd30 0x7fd43411f120 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd428009990 tx=0x7fd428009ca0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.496+0000 7fd43359e700 1 -- 172.21.15.47:0/388410444 shutdown_connections 2024-05-11T12:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.496+0000 7fd43359e700 1 --2- 172.21.15.47:0/388410444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd43411cd30 0x7fd43411f120 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.496+0000 7fd43359e700 1 --2- 172.21.15.47:0/388410444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 0x7fd43410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.496+0000 7fd43359e700 1 --2- 172.21.15.47:0/388410444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd4341120b0 0x7fd43410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.496+0000 7fd43359e700 1 -- 172.21.15.47:0/388410444 >> 172.21.15.47:0/388410444 conn(0x7fd43406e530 msgr2=0x7fd43406e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:00.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.498+0000 7fd43359e700 1 -- 172.21.15.47:0/388410444 shutdown_connections 2024-05-11T12:26:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.499+0000 7fd43359e700 1 -- 172.21.15.47:0/388410444 wait complete. 2024-05-11T12:26:00.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.501+0000 7fd43359e700 1 Processor -- start 2024-05-11T12:26:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.503+0000 7fd43359e700 1 -- start start 2024-05-11T12:26:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.503+0000 7fd43359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 0x7fd43411b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.503+0000 7fd43359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4341120b0 0x7fd43411b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.503+0000 7fd43359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43411cd30 0x7fd434114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.503+0000 7fd43359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd4341224b0 con 0x7fd4341120b0 2024-05-11T12:26:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.503+0000 7fd43359e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd434122330 con 0x7fd43411cd30 2024-05-11T12:26:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.503+0000 7fd43359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd434122630 con 0x7fd434111c40 2024-05-11T12:26:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.504+0000 7fd43259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 0x7fd43411b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.505+0000 7fd43259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 0x7fd43411b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:41050/0 (socket says 172.21.15.47:41050) 2024-05-11T12:26:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.505+0000 7fd43259c700 1 -- 172.21.15.47:0/1990568286 learned_addr learned my addr 172.21.15.47:0/1990568286 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.506+0000 7fd432d9d700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43411cd30 0x7fd434114530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.507+0000 7fd43259c700 1 -- 172.21.15.47:0/1990568286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43411cd30 msgr2=0x7fd434114530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.507+0000 7fd43259c700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43411cd30 0x7fd434114530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.507+0000 7fd43259c700 1 -- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4341120b0 msgr2=0x7fd43411b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.507+0000 7fd43259c700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4341120b0 0x7fd43411b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.507+0000 7fd43259c700 1 -- 172.21.15.47:0/1990568286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd434114e30 con 0x7fd434111c40 2024-05-11T12:26:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.508+0000 7fd43259c700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 0x7fd43411b250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00ada0 tx=0x7fd42c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.514+0000 7fd4237fe700 1 -- 172.21.15.47:0/1990568286 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c017020 con 0x7fd434111c40 2024-05-11T12:26:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.514+0000 7fd4237fe700 1 -- 172.21.15.47:0/1990568286 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42c00ce30 con 0x7fd434111c40 2024-05-11T12:26:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.514+0000 7fd4237fe700 1 -- 172.21.15.47:0/1990568286 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0055a0 con 0x7fd434111c40 2024-05-11T12:26:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.515+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4341150c0 con 0x7fd434111c40 2024-05-11T12:26:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.515+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd434119010 con 0x7fd434111c40 2024-05-11T12:26:00.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.529+0000 7fd44313eb80 1 Processor -- start 2024-05-11T12:26:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.529+0000 7fd44313eb80 1 -- start start 2024-05-11T12:26:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5d1920 0x55c4fb5cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd44313eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c4fb51d940 con 0x55c4fb5d1000 2024-05-11T12:26:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd44313eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c4fb482da0 con 0x55c4fb5d1920 2024-05-11T12:26:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd44313eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c4fb519900 con 0x55c4fb5cbb30 2024-05-11T12:26:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd432d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd432d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52592/0 (socket says 172.21.15.47:52592) 2024-05-11T12:26:00.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.530+0000 7fd432d9d700 1 -- 172.21.15.47:0/2479989698 learned_addr learned my addr 172.21.15.47:0/2479989698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:00.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.531+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.543+0000 7fd432d9d700 1 -- 172.21.15.47:0/2479989698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5cbb30 msgr2=0x55c4fb5cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.543+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.543+0000 7fd432d9d700 1 -- 172.21.15.47:0/2479989698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5d1920 msgr2=0x55c4fb5cb510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.543+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5d1920 0x55c4fb5cb510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.543+0000 7fd432d9d700 1 -- 172.21.15.47:0/2479989698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4fb46a8d0 con 0x55c4fb5d1000 2024-05-11T12:26:00.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.543+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:26:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.545+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d13e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd428009f20 tx=0x7fd428009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8fb8b4b713877d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:00.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.546+0000 7fd4237fe700 1 -- 172.21.15.47:0/1990568286 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd42c00c980 con 0x7fd434111c40 2024-05-11T12:26:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.547+0000 7fd4237fe700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd41c0b2a90 0x7fd41c0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.547+0000 7fd420ff9700 1 -- 172.21.15.47:0/2479989698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428026070 con 0x55c4fb5d1000 2024-05-11T12:26:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.548+0000 7fd420ff9700 1 -- 172.21.15.47:0/2479989698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42800b850 con 0x55c4fb5d1000 2024-05-11T12:26:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.548+0000 7fd4237fe700 1 -- 172.21.15.47:0/1990568286 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd42c0e1a40 con 0x7fd434111c40 2024-05-11T12:26:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.549+0000 7fd44313eb80 1 -- 172.21.15.47:0/2479989698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 msgr2=0x55c4fb5d13e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.549+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d13e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd428009f20 tx=0x7fd428009f50 comp rx=0 tx=0).stop 2024-05-11T12:26:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.553+0000 7fd44313eb80 1 -- 172.21.15.47:0/2479989698 shutdown_connections 2024-05-11T12:26:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.553+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.553+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d13e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.553+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2479989698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5d1920 0x55c4fb5cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.553+0000 7fd43259c700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd41c0b2a90 0x7fd41c0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.553+0000 7fd44313eb80 1 -- 172.21.15.47:0/2479989698 >> 172.21.15.47:0/2479989698 conn(0x55c4fb4f7b70 msgr2=0x55c4fb5c6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.553+0000 7fd44313eb80 1 -- 172.21.15.47:0/2479989698 shutdown_connections 2024-05-11T12:26:00.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.554+0000 7fd44313eb80 1 -- 172.21.15.47:0/2479989698 wait complete. 2024-05-11T12:26:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.555+0000 7fd44313eb80 1 -- start start 2024-05-11T12:26:00.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.555+0000 7fd44313eb80 1 Processor -- start 2024-05-11T12:26:00.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.556+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.556+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.556+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1920 0x55c4fb5d4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.556+0000 7fd44313eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c4fb5d8480 con 0x55c4fb5d1000 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.556+0000 7fd44313eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c4fb4d6ed0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.556+0000 7fd44313eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c4fb5d8600 con 0x55c4fb5d1920 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.556+0000 7fd44313eb80 10 client.0 _ll_register_callbacks cb 0x55c4fb4b5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.557+0000 7fd432d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.557+0000 7fd432d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5d7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:46638/0 (socket says 172.21.15.47:46638) 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.557+0000 7fd432d9d700 1 -- 172.21.15.47:0/2906963226 learned_addr learned my addr 172.21.15.47:0/2906963226 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:00.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.558+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1920 0x55c4fb5d4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.559+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.559+0000 7fd432d9d700 1 -- 172.21.15.47:0/2906963226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1920 msgr2=0x55c4fb5d4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:00.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.559+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1920 0x55c4fb5d4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.559+0000 7fd432d9d700 1 -- 172.21.15.47:0/2906963226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 msgr2=0x55c4fb5d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:00.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.559+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.559+0000 7fd432d9d700 1 -- 172.21.15.47:0/2906963226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4fb4d4760 con 0x55c4fb5cbb30 2024-05-11T12:26:00.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.562+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5d7660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd42800b5c0 tx=0x7fd428009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85756557a6cd663b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:00.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.563+0000 7fd419ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:26:00.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.563+0000 7fd419ffb700 1 -- 172.21.15.47:0/2906963226 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428026030 con 0x55c4fb5cbb30 2024-05-11T12:26:00.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.564+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c4fb52b320 con 0x55c4fb5cbb30 2024-05-11T12:26:00.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.564+0000 7fd419ffb700 1 -- 172.21.15.47:0/2906963226 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd42800ba60 con 0x55c4fb5cbb30 2024-05-11T12:26:00.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.564+0000 7fd419ffb700 1 -- 172.21.15.47:0/2906963226 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd42800bc60 con 0x55c4fb5cbb30 2024-05-11T12:26:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.569+0000 7fd44313eb80 10 client.24914 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:00.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.570+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c4fb5e2da0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd419ffb700 1 -- 172.21.15.47:0/2906963226 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c4fb52b320 con 0x55c4fb5cbb30 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd419ffb700 1 -- 172.21.15.47:0/2906963226 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd42800f460 con 0x55c4fb5cbb30 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd44313eb80 10 client.24914 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd44313eb80 10 client.24914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd44313eb80 20 client.24914 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5e5310 0x55c4fb5e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd44313eb80 4 client.24914 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:26:00.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.571+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4fb5253c0 con 0x55c4fb5e5310 2024-05-11T12:26:00.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.573+0000 7fd43259c700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd41c0b2a90 0x7fd41c0b4f50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0a5040 tx=0x7fd42c0a4e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:00.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.576+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5e5310 0x55c4fb5e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.589+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5e5310 0x55c4fb5e7750 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c512722569fca2 server_cookie=83215798f778ef47 in_seq=0 out_seq=0 2024-05-11T12:26:00.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.589+0000 7fd419ffb700 10 client.24914 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:00.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.595+0000 7fd419ffb700 1 -- 172.21.15.47:0/2906963226 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c4fb5253c0 con 0x55c4fb5e5310 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.595+0000 7fd419ffb700 10 client.24914 handle_command_reply: tid=0 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.595+0000 7fd44313eb80 1 client.24914 shutdown 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.596+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 msgr2=0x55c4fb5d7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.596+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5d7660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd42800b5c0 tx=0x7fd428009ca0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.596+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 shutdown_connections 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.596+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5e5310 0x55c4fb5e7750 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.596+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1920 0x55c4fb5d4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.596+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5d7660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.596+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2906963226 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.597+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 >> 172.21.15.47:0/2906963226 conn(0x55c4fb4f7b70 msgr2=0x55c4fb5c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:00.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.597+0000 7fd419ffb700 0 client.24914 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.597+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 shutdown_connections 2024-05-11T12:26:00.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.600+0000 7fd44313eb80 1 -- 172.21.15.47:0/2906963226 wait complete. 2024-05-11T12:26:00.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.601+0000 7fd44313eb80 20 client.24914 trim_cache size 0 max 16384 2024-05-11T12:26:00.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.677+0000 7fd44313eb80 1 Processor -- start 2024-05-11T12:26:00.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.678+0000 7fd44313eb80 1 -- start start 2024-05-11T12:26:00.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.679+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.679+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5e15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.679+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.680+0000 7fd44313eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd41400b720 con 0x55c4fb5d1000 2024-05-11T12:26:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.680+0000 7fd44313eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd41400dfa0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.680+0000 7fd44313eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd41400bd90 con 0x55c4fb4f7b70 2024-05-11T12:26:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.681+0000 7fd432d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.681+0000 7fd432d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:41084/0 (socket says 172.21.15.47:41084) 2024-05-11T12:26:00.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.681+0000 7fd432d9d700 1 -- 172.21.15.47:0/1506687284 learned_addr learned my addr 172.21.15.47:0/1506687284 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:00.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.685+0000 7fd431d9b700 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5e15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.686+0000 7fd432d9d700 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.689+0000 7fd432d9d700 1 -- 172.21.15.47:0/1506687284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 msgr2=0x55c4fb5e15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:00.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.689+0000 7fd432d9d700 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5e15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.689+0000 7fd432d9d700 1 -- 172.21.15.47:0/1506687284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 msgr2=0x55c4fb5d6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:00.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.689+0000 7fd432d9d700 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.689+0000 7fd432d9d700 1 -- 172.21.15.47:0/1506687284 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4fb5c7ea0 con 0x55c4fb4f7b70 2024-05-11T12:26:00.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.690+0000 7fd432d9d700 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5cb510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd428009d90 tx=0x7fd428005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5e2b61636b93fd4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.690+0000 7fd41bfff700 1 -- 172.21.15.47:0/1506687284 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428026040 con 0x55c4fb4f7b70 2024-05-11T12:26:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.690+0000 7fd41bfff700 1 -- 172.21.15.47:0/1506687284 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd42800fbd0 con 0x55c4fb4f7b70 2024-05-11T12:26:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.691+0000 7fd41bfff700 1 -- 172.21.15.47:0/1506687284 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42803bc20 con 0x55c4fb4f7b70 2024-05-11T12:26:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.692+0000 7fd44313eb80 1 -- 172.21.15.47:0/1506687284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb4f7b70 msgr2=0x55c4fb5cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.692+0000 7fd44313eb80 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5cb510 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd428009d90 tx=0x7fd428005fb0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.693+0000 7fd44313eb80 1 -- 172.21.15.47:0/1506687284 shutdown_connections 2024-05-11T12:26:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.693+0000 7fd44313eb80 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb5d1000 0x55c4fb5d6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.693+0000 7fd44313eb80 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5cb510 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.693+0000 7fd44313eb80 1 --2- 172.21.15.47:0/1506687284 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5e15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.693+0000 7fd44313eb80 1 -- 172.21.15.47:0/1506687284 >> 172.21.15.47:0/1506687284 conn(0x55c4fb5cc270 msgr2=0x55c4fb5d23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:00.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.695+0000 7fd44313eb80 1 -- 172.21.15.47:0/1506687284 shutdown_connections 2024-05-11T12:26:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.695+0000 7fd44313eb80 1 -- 172.21.15.47:0/1506687284 wait complete. 2024-05-11T12:26:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.696+0000 7fd44313eb80 1 -- start start 2024-05-11T12:26:00.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.696+0000 7fd44313eb80 1 Processor -- start 2024-05-11T12:26:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.698+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.698+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.698+0000 7fd44313eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1000 0x55c4fb647ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.698+0000 7fd44313eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd41400c460 con 0x55c4fb4f7b70 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.699+0000 7fd44313eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd41400d8d0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.699+0000 7fd431d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.699+0000 7fd431d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5db3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:46656/0 (socket says 172.21.15.47:46656) 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.699+0000 7fd431d9b700 1 -- 172.21.15.47:0/2400728471 learned_addr learned my addr 172.21.15.47:0/2400728471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.699+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd41400cb30 con 0x55c4fb5d1000 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.700+0000 7fd44313eb80 10 client.0 _ll_register_callbacks cb 0x55c4fb4b5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.700+0000 7fd431d9b700 1 -- 172.21.15.47:0/2400728471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1000 msgr2=0x55c4fb647ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.700+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1000 0x55c4fb647ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.700+0000 7fd431d9b700 1 -- 172.21.15.47:0/2400728471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb4f7b70 msgr2=0x55c4fb5dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:00.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.700+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.700+0000 7fd431d9b700 1 -- 172.21.15.47:0/2400728471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4fb6481e0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.710+0000 7fd431d9b700 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5db3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55c4fb5d5fd0 tx=0x7fd42400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=211197958fa7f6ef server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:00.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.711+0000 7fd419ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:26:00.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.716+0000 7fd419ffb700 1 -- 172.21.15.47:0/2400728471 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424019710 con 0x55c4fb5cbb30 2024-05-11T12:26:00.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.716+0000 7fd419ffb700 1 -- 172.21.15.47:0/2400728471 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd4240198b0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.717+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd42800fbd0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.721+0000 7fd419ffb700 1 -- 172.21.15.47:0/2400728471 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd424019ab0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.721+0000 7fd44313eb80 10 client.24920 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:00.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.721+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c4fb64f5d0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd419ffb700 1 -- 172.21.15.47:0/2400728471 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd424010e30 con 0x55c4fb5cbb30 2024-05-11T12:26:00.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd419ffb700 1 -- 172.21.15.47:0/2400728471 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd4240215f0 con 0x55c4fb5cbb30 2024-05-11T12:26:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd44313eb80 10 client.24920 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd44313eb80 10 client.24920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd44313eb80 20 client.24920 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5d1920 0x55c4fb651c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd44313eb80 4 client.24920 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:26:00.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.727+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd42800fbd0 con 0x55c4fb5d1920 2024-05-11T12:26:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.729+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5d1920 0x55c4fb651c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.730+0000 7fd432d9d700 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5d1920 0x55c4fb651c60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2597ec959c38d79 server_cookie=4215161ae4693d8a in_seq=0 out_seq=0 2024-05-11T12:26:00.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.730+0000 7fd419ffb700 10 client.24920 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.735+0000 7fd419ffb700 1 -- 172.21.15.47:0/2400728471 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fd42800fbd0 con 0x55c4fb5d1920 2024-05-11T12:26:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.735+0000 7fd419ffb700 10 client.24920 handle_command_reply: tid=0 2024-05-11T12:26:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.736+0000 7fd44313eb80 1 client.24920 shutdown 2024-05-11T12:26:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.738+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 msgr2=0x55c4fb5db3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.738+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5db3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x55c4fb5d5fd0 tx=0x7fd42400bf90 comp rx=0 tx=0).stop 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.744+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 shutdown_connections 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.745+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c4fb5d1920 0x55c4fb651c60 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.745+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c4fb5d1000 0x55c4fb647ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.745+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c4fb4f7b70 0x55c4fb5dae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.745+0000 7fd44313eb80 1 --2- 172.21.15.47:0/2400728471 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c4fb5cbb30 0x55c4fb5db3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.745+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 >> 172.21.15.47:0/2400728471 conn(0x55c4fb5cc270 msgr2=0x55c4fb5c5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:00.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.745+0000 7fd419ffb700 0 client.24920 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:00.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.746+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 shutdown_connections 2024-05-11T12:26:00.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.746+0000 7fd44313eb80 1 -- 172.21.15.47:0/2400728471 wait complete. 2024-05-11T12:26:00.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.746+0000 7fd44313eb80 20 client.24920 trim_cache size 0 max 16384 2024-05-11T12:26:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd41c0b2a90 msgr2=0x7fd41c0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd41c0b2a90 0x7fd41c0b4f50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0a5040 tx=0x7fd42c0a4e80 comp rx=0 tx=0).stop 2024-05-11T12:26:00.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 msgr2=0x7fd43411b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 0x7fd43411b250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00ada0 tx=0x7fd42c00c330 comp rx=0 tx=0).stop 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 shutdown_connections 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd41c0b2a90 0x7fd41c0b4f50 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0a5040 tx=0x7fd42c0a4e80 comp rx=0 tx=0).stop 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd43411cd30 0x7fd434114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd434111c40 0x7fd43411b250 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00ada0 tx=0x7fd42c00c330 comp rx=0 tx=0).stop 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 --2- 172.21.15.47:0/1990568286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd4341120b0 0x7fd43411b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.748+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 >> 172.21.15.47:0/1990568286 conn(0x7fd43406e530 msgr2=0x7fd434071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.750+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 shutdown_connections 2024-05-11T12:26:00.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:00.750+0000 7fd43359e700 1 -- 172.21.15.47:0/1990568286 wait complete. 2024-05-11T12:26:01.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:00 smithi047 ceph-mon[30375]: pgmap v442: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 648 MiB/s wr, 805 op/s 2024-05-11T12:26:01.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:00 smithi064 ceph-mon[34537]: pgmap v442: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 648 MiB/s wr, 805 op/s 2024-05-11T12:26:01.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:00 smithi050 ceph-mon[34072]: pgmap v442: 97 pgs: 97 active+clean; 7.2 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 648 MiB/s wr, 805 op/s 2024-05-11T12:26:03.012 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:26:03.013 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:26:03.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:02 smithi064 ceph-mon[34537]: pgmap v443: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 758 MiB/s wr, 1.17k op/s 2024-05-11T12:26:03.120 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:02 smithi047 ceph-mon[30375]: pgmap v443: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 758 MiB/s wr, 1.17k op/s 2024-05-11T12:26:03.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:02 smithi050 ceph-mon[34072]: pgmap v443: 97 pgs: 97 active+clean; 8.2 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 758 MiB/s wr, 1.17k op/s 2024-05-11T12:26:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:26:05.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:04 smithi047 ceph-mon[30375]: pgmap v444: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 1.71k op/s 2024-05-11T12:26:05.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:04 smithi064 ceph-mon[34537]: pgmap v444: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 1.71k op/s 2024-05-11T12:26:05.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:04 smithi050 ceph-mon[34072]: pgmap v444: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 1.71k op/s 2024-05-11T12:26:07.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:06 smithi064 ceph-mon[34537]: pgmap v445: 97 pgs: 97 active+clean; 9.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.83k op/s 2024-05-11T12:26:07.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:06 smithi050 ceph-mon[34072]: pgmap v445: 97 pgs: 97 active+clean; 9.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.83k op/s 2024-05-11T12:26:07.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:06 smithi047 ceph-mon[30375]: pgmap v445: 97 pgs: 97 active+clean; 9.6 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 778 MiB/s wr, 1.83k op/s 2024-05-11T12:26:08.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.676+0000 7f361fe87700 1 -- 172.21.15.47:0/2765084420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f361811ca70 msgr2=0x7f361811ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:08.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.676+0000 7f361fe87700 1 --2- 172.21.15.47:0/2765084420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f361811ca70 0x7f361811ee60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f361000b0a0 tx=0x7f361000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:26:08.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.684+0000 7f361fe87700 1 -- 172.21.15.47:0/2765084420 shutdown_connections 2024-05-11T12:26:08.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.684+0000 7f361fe87700 1 --2- 172.21.15.47:0/2765084420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f361811ca70 0x7f361811ee60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:08.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.684+0000 7f361fe87700 1 --2- 172.21.15.47:0/2765084420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36180840e0 0x7f361807bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:08.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.684+0000 7f361fe87700 1 --2- 172.21.15.47:0/2765084420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361807c410 0x7f361807c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:08.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.684+0000 7f361fe87700 1 -- 172.21.15.47:0/2765084420 >> 172.21.15.47:0/2765084420 conn(0x7f3618076d20 msgr2=0x7f3618079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:08.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.686+0000 7f361fe87700 1 -- 172.21.15.47:0/2765084420 shutdown_connections 2024-05-11T12:26:08.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.687+0000 7f361fe87700 1 -- 172.21.15.47:0/2765084420 wait complete. 2024-05-11T12:26:08.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.688+0000 7f361fe87700 1 Processor -- start 2024-05-11T12:26:08.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.692+0000 7f361fe87700 1 -- start start 2024-05-11T12:26:08.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.692+0000 7f361fe87700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361807c410 0x7f36181ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:08.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.692+0000 7f361fe87700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180840e0 0x7f36181aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:08.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.692+0000 7f361fe87700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361811ca70 0x7f36181a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:08.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.692+0000 7f361fe87700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3618122610 con 0x7f361807c410 2024-05-11T12:26:08.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.692+0000 7f361fe87700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3618122490 con 0x7f36180840e0 2024-05-11T12:26:08.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.692+0000 7f361fe87700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3618122790 con 0x7f361811ca70 2024-05-11T12:26:08.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.693+0000 7f361dc23700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361807c410 0x7f36181ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:08.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.693+0000 7f361dc23700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361807c410 0x7f36181ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43808/0 (socket says 172.21.15.47:43808) 2024-05-11T12:26:08.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.693+0000 7f361dc23700 1 -- 172.21.15.47:0/1677891958 learned_addr learned my addr 172.21.15.47:0/1677891958 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:08.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.693+0000 7f361e424700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361811ca70 0x7f36181a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:08.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.697+0000 7f361d422700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180840e0 0x7f36181aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:08.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.697+0000 7f361dc23700 1 -- 172.21.15.47:0/1677891958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361811ca70 msgr2=0x7f36181a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:08.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.697+0000 7f361dc23700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361811ca70 0x7f36181a8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:08.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.697+0000 7f361dc23700 1 -- 172.21.15.47:0/1677891958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180840e0 msgr2=0x7f36181aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:08.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.697+0000 7f361dc23700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180840e0 0x7f36181aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:08.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.697+0000 7f361dc23700 1 -- 172.21.15.47:0/1677891958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36181a90b0 con 0x7f361807c410 2024-05-11T12:26:08.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.697+0000 7f361d422700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180840e0 0x7f36181aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:26:08.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.698+0000 7f361dc23700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361807c410 0x7f36181ae740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f360800b550 tx=0x7f360800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:08.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.700+0000 7f360effd700 1 -- 172.21.15.47:0/1677891958 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3608015020 con 0x7f361807c410 2024-05-11T12:26:08.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.700+0000 7f360effd700 1 -- 172.21.15.47:0/1677891958 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3608011ce0 con 0x7f361807c410 2024-05-11T12:26:08.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.700+0000 7f360effd700 1 -- 172.21.15.47:0/1677891958 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3608010db0 con 0x7f361807c410 2024-05-11T12:26:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.701+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36181a93a0 con 0x7f361807c410 2024-05-11T12:26:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.701+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361806cf70 con 0x7f361807c410 2024-05-11T12:26:08.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.709+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35fc005330 con 0x7f361807c410 2024-05-11T12:26:08.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.711+0000 7f360effd700 1 -- 172.21.15.47:0/1677891958 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3608018020 con 0x7f361807c410 2024-05-11T12:26:08.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.712+0000 7f360effd700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36040b29d0 0x7f36040b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:08.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.713+0000 7f360effd700 1 -- 172.21.15.47:0/1677891958 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f36080e1950 con 0x7f361807c410 2024-05-11T12:26:08.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.714+0000 7f361d422700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36040b29d0 0x7f36040b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:08.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.715+0000 7f361d422700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36040b29d0 0x7f36040b4e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f361811eeb0 tx=0x7f3614009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:08.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:08.731+0000 7f360effd700 1 -- 172.21.15.47:0/1677891958 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f360801f020 con 0x7f361807c410 2024-05-11T12:26:09.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:08 smithi047 ceph-mon[30375]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 2.23k op/s 2024-05-11T12:26:09.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:08 smithi050 ceph-mon[34072]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 2.23k op/s 2024-05-11T12:26:09.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.342+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35fc005620 con 0x7f361807c410 2024-05-11T12:26:09.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.352+0000 7f360effd700 1 -- 172.21.15.47:0/1677891958 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f36080a95a0 con 0x7f361807c410 2024-05-11T12:26:09.352 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:26:09.353 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:26:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.357+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36040b29d0 msgr2=0x7f36040b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.357+0000 7f361fe87700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36040b29d0 0x7f36040b4e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f361811eeb0 tx=0x7f3614009260 comp rx=0 tx=0).stop 2024-05-11T12:26:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.357+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361807c410 msgr2=0x7f36181ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:09.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.357+0000 7f361fe87700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361807c410 0x7f36181ae740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f360800b550 tx=0x7f360800b860 comp rx=0 tx=0).stop 2024-05-11T12:26:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.361+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 shutdown_connections 2024-05-11T12:26:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.361+0000 7f361fe87700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f361811ca70 0x7f36181a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.361+0000 7f361fe87700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f361807c410 0x7f36181ae740 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.361+0000 7f361fe87700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36040b29d0 0x7f36040b4e90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.361+0000 7f361fe87700 1 --2- 172.21.15.47:0/1677891958 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f36180840e0 0x7f36181aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:09.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.361+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 >> 172.21.15.47:0/1677891958 conn(0x7f3618076d20 msgr2=0x7f3618082190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:09.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:08 smithi064 ceph-mon[34537]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 2.23k op/s 2024-05-11T12:26:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.364+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 shutdown_connections 2024-05-11T12:26:09.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:09.364+0000 7f361fe87700 1 -- 172.21.15.47:0/1677891958 wait complete. 2024-05-11T12:26:09.372 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:26:10.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:09 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1677891958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:26:10.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:09 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1677891958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:26:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:09 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1677891958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:26:11.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:10 smithi050 ceph-mon[34072]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 754 MiB/s wr, 1.94k op/s 2024-05-11T12:26:11.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:10 smithi047 ceph-mon[30375]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 754 MiB/s wr, 1.94k op/s 2024-05-11T12:26:11.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:10 smithi064 ceph-mon[34537]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 754 MiB/s wr, 1.94k op/s 2024-05-11T12:26:13.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:26:13.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:12 smithi050 ceph-mon[34072]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 804 MiB/s wr, 2.06k op/s 2024-05-11T12:26:13.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:12 smithi047 ceph-mon[30375]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 804 MiB/s wr, 2.06k op/s 2024-05-11T12:26:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:12 smithi064 ceph-mon[34537]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 804 MiB/s wr, 2.06k op/s 2024-05-11T12:26:14.099 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:26:15.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:14 smithi050 ceph-mon[34072]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 896 MiB/s wr, 2.16k op/s 2024-05-11T12:26:15.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:14 smithi047 ceph-mon[30375]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 896 MiB/s wr, 2.16k op/s 2024-05-11T12:26:15.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:14 smithi064 ceph-mon[34537]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 896 MiB/s wr, 2.16k op/s 2024-05-11T12:26:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:16 smithi047 ceph-mon[30375]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 1.81k op/s 2024-05-11T12:26:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:16 smithi064 ceph-mon[34537]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 1.81k op/s 2024-05-11T12:26:17.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:16 smithi050 ceph-mon[34072]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 1.81k op/s 2024-05-11T12:26:19.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:18 smithi047 ceph-mon[30375]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 927 MiB/s wr, 2.17k op/s 2024-05-11T12:26:19.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:18 smithi064 ceph-mon[34537]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 927 MiB/s wr, 2.17k op/s 2024-05-11T12:26:19.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:18 smithi050 ceph-mon[34072]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 927 MiB/s wr, 2.17k op/s 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.602+0000 7f0ba3fff700 1 -- 172.21.15.47:0/2900223197 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac06ac40 con 0x7f0ba40b7b00 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.603+0000 7f0bb25c1700 1 -- 172.21.15.47:0/2900223197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ba40b7b00 msgr2=0x7f0ba40b9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.603+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/2900223197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ba40b7b00 0x7f0ba40b9ef0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0bac068fd0 tx=0x7f0bac069b60 comp rx=0 tx=0).stop 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.609+0000 7f0bb25c1700 1 -- 172.21.15.47:0/2900223197 shutdown_connections 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.609+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/2900223197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ba40b7b00 0x7f0ba40b9ef0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.609+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/2900223197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 0x7f0ba40a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.609+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/2900223197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ba40a5700 0x7f0ba40b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.609+0000 7f0bb25c1700 1 -- 172.21.15.47:0/2900223197 >> 172.21.15.47:0/2900223197 conn(0x7f0ba401a6a0 msgr2=0x7f0ba401aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:19.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.615+0000 7f0bb25c1700 1 -- 172.21.15.47:0/2900223197 shutdown_connections 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.618+0000 7f0bb25c1700 1 -- 172.21.15.47:0/2900223197 wait complete. 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.619+0000 7f0bb25c1700 1 Processor -- start 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.628+0000 7f0bb25c1700 1 -- start start 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.628+0000 7f0bb25c1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 0x7f0ba40b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.629+0000 7f0bb25c1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ba40a5700 0x7f0ba40aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.629+0000 7f0bb25c1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ba40b7b00 0x7f0ba40af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.629+0000 7f0bb25c1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40bd540 con 0x7f0ba40a5700 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.629+0000 7f0bb25c1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40bd3c0 con 0x7f0ba40a4de0 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.629+0000 7f0bb25c1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba40bd6c0 con 0x7f0ba40b7b00 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.631+0000 7f0bb15bf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 0x7f0ba40b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.631+0000 7f0bb15bf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 0x7f0ba40b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:49322/0 (socket says 172.21.15.47:49322) 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.631+0000 7f0bb15bf700 1 -- 172.21.15.47:0/4021409003 learned_addr learned my addr 172.21.15.47:0/4021409003 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.632+0000 7f0bb15bf700 1 -- 172.21.15.47:0/4021409003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ba40b7b00 msgr2=0x7f0ba40af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.632+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ba40b7b00 0x7f0ba40af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.632+0000 7f0bb15bf700 1 -- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ba40a5700 msgr2=0x7f0ba40aec10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.632+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ba40a5700 0x7f0ba40aec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.632+0000 7f0bb15bf700 1 -- 172.21.15.47:0/4021409003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40af790 con 0x7f0ba40a4de0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.633+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 0x7f0ba40b5b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800b550 tx=0x7f0ba800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.635+0000 7f0ba27fc700 1 -- 172.21.15.47:0/4021409003 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8015020 con 0x7f0ba40a4de0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.635+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba40b36b0 con 0x7f0ba40a4de0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.635+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba40b3bf0 con 0x7f0ba40a4de0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.636+0000 7f0ba27fc700 1 -- 172.21.15.47:0/4021409003 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ba8011ce0 con 0x7f0ba40a4de0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.636+0000 7f0ba27fc700 1 -- 172.21.15.47:0/4021409003 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8010d50 con 0x7f0ba40a4de0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.647+0000 7f0ba27fc700 1 -- 172.21.15.47:0/4021409003 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0ba8018020 con 0x7f0ba40a4de0 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.647+0000 7f0bbe0b9b80 1 Processor -- start 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.648+0000 7f0bbe0b9b80 1 -- start start 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.648+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d0722d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.648+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722dba0 0x564d07231810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.648+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d07232190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.648+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564d0717f860 con 0x564d0722dba0 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.648+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564d0711e7c0 con 0x564d0722d280 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.648+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564d0717b820 con 0x564d07231d50 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.649+0000 7f0bb1dc0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d0722d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.649+0000 7f0bb1dc0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d0722d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:49338/0 (socket says 172.21.15.47:49338) 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.649+0000 7f0bb1dc0700 1 -- 172.21.15.47:0/491347140 learned_addr learned my addr 172.21.15.47:0/491347140 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.649+0000 7f0bb1dc0700 1 -- 172.21.15.47:0/491347140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 msgr2=0x564d07232190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.649+0000 7f0bb1dc0700 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d07232190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.649+0000 7f0bb1dc0700 1 -- 172.21.15.47:0/491347140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722dba0 msgr2=0x564d07231810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.649+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722dba0 0x564d07231810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.650+0000 7f0bb1dc0700 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722dba0 0x564d07231810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.650+0000 7f0bb1dc0700 1 -- 172.21.15.47:0/491347140 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d070cc8d0 con 0x564d0722d280 2024-05-11T12:26:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.650+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722dba0 0x564d07231810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.651+0000 7f0bb1dc0700 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d0722d660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0bac052090 tx=0x7f0bac068f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5dee3e7016b9c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.652+0000 7f0b977fe700 1 -- 172.21.15.47:0/491347140 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac07c040 con 0x564d0722d280 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.652+0000 7f0b977fe700 1 -- 172.21.15.47:0/491347140 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bac06cb40 con 0x564d0722d280 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.652+0000 7f0b977fe700 1 -- 172.21.15.47:0/491347140 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac06ac70 con 0x564d0722d280 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.655+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/491347140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 msgr2=0x564d0722d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.655+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d0722d660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0bac052090 tx=0x7f0bac068f10 comp rx=0 tx=0).stop 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.656+0000 7f0ba27fc700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0b980b2a90 0x7f0b980b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.657+0000 7f0bb1dc0700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0b980b2a90 0x7f0b980b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.657+0000 7f0ba27fc700 1 -- 172.21.15.47:0/4021409003 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0ba80e1b10 con 0x7f0ba40a4de0 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.658+0000 7f0bb1dc0700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0b980b2a90 0x7f0b980b4f50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06c040 tx=0x7f0bac06ae90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.669+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/491347140 shutdown_connections 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.669+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d07232190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.669+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d0722d660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.669+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/491347140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722dba0 0x564d07231810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.669+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/491347140 >> 172.21.15.47:0/491347140 conn(0x564d07159e60 msgr2=0x564d07227cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.672+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/491347140 shutdown_connections 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.679+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/491347140 wait complete. 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.680+0000 7f0bbe0b9b80 1 -- start start 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.680+0000 7f0bbe0b9b80 1 Processor -- start 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.682+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d07239200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.682+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d072365a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.689+0000 7f0bb15bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d07239200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.689+0000 7f0bb15bf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d07239200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56884/0 (socket says 172.21.15.47:56884) 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.689+0000 7f0bb15bf700 1 -- 172.21.15.47:0/3115641412 learned_addr learned my addr 172.21.15.47:0/3115641412 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.689+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d07231d50 0x564d07236ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.689+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564d07239ef0 con 0x564d0722d280 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.689+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564d070e5c30 con 0x564d07231d50 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.690+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564d0723a070 con 0x564d0722dba0 2024-05-11T12:26:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.690+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d072365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.690+0000 7f0bbe0b9b80 10 client.0 _ll_register_callbacks cb 0x564d07117920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.690+0000 7f0bb1dc0700 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d07231d50 0x564d07236ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.691+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d07231d50 msgr2=0x564d07236ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.691+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d07231d50 0x564d07236ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.691+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/3115641412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 msgr2=0x564d07239200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.691+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d07239200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.691+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/3115641412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0711b690 con 0x564d0722dba0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.692+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d072365a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40b0460 tx=0x7f0b9c00e6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b799ee9ad7d1f5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.693+0000 7f0b94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.695+0000 7f0b94ff9700 1 -- 172.21.15.47:0/3115641412 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004230 con 0x564d0722dba0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.695+0000 7f0b94ff9700 1 -- 172.21.15.47:0/3115641412 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0b9c0043d0 con 0x564d0722dba0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.695+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bac06cb40 con 0x564d0722dba0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.696+0000 7f0b94ff9700 1 -- 172.21.15.47:0/3115641412 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0b9c003680 con 0x564d0722dba0 2024-05-11T12:26:19.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.696+0000 7f0bbe0b9b80 10 client.24967 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.696+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564d072447a0 con 0x564d0722dba0 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.700+0000 7f0b94ff9700 1 -- 172.21.15.47:0/3115641412 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b9c003cd0 con 0x564d0722dba0 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.700+0000 7f0b94ff9700 1 -- 172.21.15.47:0/3115641412 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0b9c0239d0 con 0x564d0722dba0 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.702+0000 7f0bbe0b9b80 10 client.24967 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.702+0000 7f0bbe0b9b80 10 client.24967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.702+0000 7f0bbe0b9b80 20 client.24967 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.702+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d07246d30 0x564d07249120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.702+0000 7f0bbe0b9b80 4 client.24967 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.702+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0bac06cb40 con 0x564d07246d30 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.705+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d07246d30 0x564d07249120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.712+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d07246d30 0x564d07249120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8023eb7e6be5db67 server_cookie=1f09081f17c4d918 in_seq=0 out_seq=0 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.712+0000 7f0b94ff9700 10 client.24967 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.714+0000 7f0b94ff9700 1 -- 172.21.15.47:0/3115641412 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f0bac06cb40 con 0x564d07246d30 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.714+0000 7f0b94ff9700 10 client.24967 handle_command_reply: tid=0 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.714+0000 7f0bbe0b9b80 1 client.24967 shutdown 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.716+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 msgr2=0x564d072365a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.716+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d072365a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40b0460 tx=0x7f0b9c00e6e0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.720+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 shutdown_connections 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.720+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d07246d30 0x564d07249120 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.720+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d07231d50 0x564d07236ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.720+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d07239200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.720+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/3115641412 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d072365a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.720+0000 7f0b94ff9700 0 client.24967 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.720+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 >> 172.21.15.47:0/3115641412 conn(0x564d07159e60 msgr2=0x564d07227bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.732+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 shutdown_connections 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.732+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/3115641412 wait complete. 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.735+0000 7f0bbe0b9b80 20 client.24967 trim_cache size 0 max 16384 2024-05-11T12:26:19.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.767+0000 7f0bbe0b9b80 1 Processor -- start 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.769+0000 7f0bbe0b9b80 1 -- start start 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.769+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d07235ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.769+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d07230e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.769+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d07231d50 0x564d07238bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.769+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9000b7e0 con 0x564d07231d50 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.769+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9000e060 con 0x564d0722d280 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.769+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9000be50 con 0x564d0722dba0 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb1dc0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d07231d50 0x564d07238bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d07230e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d07230e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:33888/0 (socket says 172.21.15.47:33888) 2024-05-11T12:26:19.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/1961953281 learned_addr learned my addr 172.21.15.47:0/1961953281 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d07235ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/1961953281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 msgr2=0x564d07235ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d07235ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/1961953281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d07231d50 msgr2=0x564d07238bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d07231d50 0x564d07238bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.770+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/1961953281 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d07222240 con 0x564d0722dba0 2024-05-11T12:26:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.771+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d07230e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c019500 tx=0x7f0b9c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916040c046f799cf server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.773+0000 7f0b96ffd700 1 -- 172.21.15.47:0/1961953281 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c029070 con 0x564d0722dba0 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.773+0000 7f0b96ffd700 1 -- 172.21.15.47:0/1961953281 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b9c019d00 con 0x564d0722dba0 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.773+0000 7f0b96ffd700 1 -- 172.21.15.47:0/1961953281 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c023880 con 0x564d0722dba0 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.774+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1961953281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 msgr2=0x564d07230e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.774+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d07230e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c019500 tx=0x7f0b9c005b00 comp rx=0 tx=0).stop 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.774+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1961953281 shutdown_connections 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.774+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d07231d50 0x564d07238bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.774+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722d280 0x564d07235ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.774+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1961953281 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d0722dba0 0x564d07230e20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.775+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1961953281 >> 172.21.15.47:0/1961953281 conn(0x564d07159e60 msgr2=0x564d07230710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.777+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1961953281 shutdown_connections 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.777+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1961953281 wait complete. 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.777+0000 7f0bbe0b9b80 1 -- start start 2024-05-11T12:26:19.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.778+0000 7f0bbe0b9b80 1 Processor -- start 2024-05-11T12:26:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.780+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d072a8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.780+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722dba0 0x564d072a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.780+0000 7f0bbe0b9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d0723c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.780+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9000c520 con 0x564d0722d280 2024-05-11T12:26:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.780+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9000d990 con 0x564d0722dba0 2024-05-11T12:26:19.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.780+0000 7f0bbe0b9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9000cbf0 con 0x564d07231d50 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.780+0000 7f0bbe0b9b80 10 client.0 _ll_register_callbacks cb 0x564d07117920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.782+0000 7f0bb1dc0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d0723c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.782+0000 7f0bb1dc0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d0723c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:33898/0 (socket says 172.21.15.47:33898) 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.782+0000 7f0bb1dc0700 1 -- 172.21.15.47:0/1524603909 learned_addr learned my addr 172.21.15.47:0/1524603909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.782+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722dba0 0x564d072a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.782+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d072a8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.783+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/1524603909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 msgr2=0x564d0723c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.783+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d0723c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.783+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/1524603909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 msgr2=0x564d072a8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.783+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d072a8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:19.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.783+0000 7f0bb0dbe700 1 -- 172.21.15.47:0/1524603909 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0723d120 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.784+0000 7f0bb0dbe700 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722dba0 0x564d072a8db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e720 tx=0x7f0b9c00eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec658446482ffb5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.784+0000 7f0b94ff9700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.784+0000 7f0b94ff9700 1 -- 172.21.15.47:0/1524603909 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c029030 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.784+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b9c019d00 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.785+0000 7f0b94ff9700 1 -- 172.21.15.47:0/1524603909 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0b9c023690 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.785+0000 7f0b94ff9700 1 -- 172.21.15.47:0/1524603909 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0b9c0196d0 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.785+0000 7f0bbe0b9b80 10 client.24938 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.785+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564d072b10a0 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0b94ff9700 1 -- 172.21.15.47:0/1524603909 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b9c019d00 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0b94ff9700 1 -- 172.21.15.47:0/1524603909 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0b9c010430 con 0x564d0722dba0 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0bbe0b9b80 10 client.24938 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0bbe0b9b80 10 client.24938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:19.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0bbe0b9b80 20 client.24938 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d072b16c0 0x564d072b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0bbe0b9b80 4 client.24938 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:26:19.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.786+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d0718d240 con 0x564d072b16c0 2024-05-11T12:26:19.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.893+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d072b16c0 0x564d072b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:19.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.900+0000 7f0bb15bf700 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d072b16c0 0x564d072b3ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23b77339b2fe630e server_cookie=6c55870694d312bf in_seq=0 out_seq=0 2024-05-11T12:26:19.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:19.900+0000 7f0b94ff9700 10 client.24938 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:20.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.112+0000 7f0b94ff9700 1 -- 172.21.15.47:0/1524603909 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d0718d240 con 0x564d072b16c0 2024-05-11T12:26:20.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.112+0000 7f0b94ff9700 10 client.24938 handle_command_reply: tid=0 2024-05-11T12:26:20.114 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.112+0000 7f0bbe0b9b80 1 client.24938 shutdown 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722dba0 msgr2=0x564d072a8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722dba0 0x564d072a8db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e720 tx=0x7f0b9c00eae0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 shutdown_connections 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564d072b16c0 0x564d072b3ae0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564d07231d50 0x564d0723c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564d0722d280 0x564d072a8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 --2- 172.21.15.47:0/1524603909 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564d0722dba0 0x564d072a8db0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 >> 172.21.15.47:0/1524603909 conn(0x564d07159e60 msgr2=0x564d0722c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0b94ff9700 0 client.24938 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 shutdown_connections 2024-05-11T12:26:20.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 1 -- 172.21.15.47:0/1524603909 wait complete. 2024-05-11T12:26:20.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.113+0000 7f0bbe0b9b80 20 client.24938 trim_cache size 0 max 16384 2024-05-11T12:26:20.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.115+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0b980b2a90 msgr2=0x7f0b980b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.115+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0b980b2a90 0x7f0b980b4f50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0bac06c040 tx=0x7f0bac06ae90 comp rx=0 tx=0).stop 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.115+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 msgr2=0x7f0ba40b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.115+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 0x7f0ba40b5b80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800b550 tx=0x7f0ba800b910 comp rx=0 tx=0).stop 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.116+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 shutdown_connections 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.116+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ba40b7b00 0x7f0ba40af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.116+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0b980b2a90 0x7f0b980b4f50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.116+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ba40a4de0 0x7f0ba40b5b80 secure :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800b550 tx=0x7f0ba800b910 comp rx=0 tx=0).stop 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.116+0000 7f0bb25c1700 1 --2- 172.21.15.47:0/4021409003 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ba40a5700 0x7f0ba40aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.116+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 >> 172.21.15.47:0/4021409003 conn(0x7f0ba401a6a0 msgr2=0x7f0ba40a1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:20.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.117+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 shutdown_connections 2024-05-11T12:26:20.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:20.117+0000 7f0bb25c1700 1 -- 172.21.15.47:0/4021409003 wait complete. 2024-05-11T12:26:21.290 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:20 smithi047 ceph-mon[30375]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 MiB/s wr, 1.90k op/s 2024-05-11T12:26:21.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:20 smithi064 ceph-mon[34537]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 MiB/s wr, 1.90k op/s 2024-05-11T12:26:21.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:20 smithi050 ceph-mon[34072]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 MiB/s wr, 1.90k op/s 2024-05-11T12:26:23.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:22 smithi047 ceph-mon[30375]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 818 MiB/s wr, 2.11k op/s 2024-05-11T12:26:23.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:22 smithi064 ceph-mon[34537]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 818 MiB/s wr, 2.11k op/s 2024-05-11T12:26:23.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:22 smithi050 ceph-mon[34072]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 818 MiB/s wr, 2.11k op/s 2024-05-11T12:26:23.866 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:26:23.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:26:24.866 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:26:25.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:25 smithi047 ceph-mon[30375]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 886 MiB/s wr, 2.38k op/s 2024-05-11T12:26:25.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:25 smithi064 ceph-mon[34537]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 886 MiB/s wr, 2.38k op/s 2024-05-11T12:26:25.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:25 smithi050 ceph-mon[34072]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 886 MiB/s wr, 2.38k op/s 2024-05-11T12:26:27.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:27 smithi064 ceph-mon[34537]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 765 MiB/s wr, 2.11k op/s 2024-05-11T12:26:27.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:27 smithi050 ceph-mon[34072]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 765 MiB/s wr, 2.11k op/s 2024-05-11T12:26:27.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:27 smithi047 ceph-mon[30375]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 765 MiB/s wr, 2.11k op/s 2024-05-11T12:26:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:29 smithi064 ceph-mon[34537]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 866 MiB/s wr, 2.39k op/s 2024-05-11T12:26:29.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:29 smithi050 ceph-mon[34072]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 866 MiB/s wr, 2.39k op/s 2024-05-11T12:26:29.451 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:29 smithi047 ceph-mon[30375]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 866 MiB/s wr, 2.39k op/s 2024-05-11T12:26:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.391+0000 7fefbd21d700 1 -- 172.21.15.47:0/934362481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 msgr2=0x7fefb8116550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.391+0000 7fefbd21d700 1 --2- 172.21.15.47:0/934362481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 0x7fefb8116550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fefac0098f0 tx=0x7fefac009c00 comp rx=0 tx=0).stop 2024-05-11T12:26:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.394+0000 7fefbd21d700 1 -- 172.21.15.47:0/934362481 shutdown_connections 2024-05-11T12:26:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.394+0000 7fefbd21d700 1 --2- 172.21.15.47:0/934362481 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 0x7fefb8116550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.394+0000 7fefbd21d700 1 --2- 172.21.15.47:0/934362481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 0x7fefb81018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.394+0000 7fefbd21d700 1 --2- 172.21.15.47:0/934362481 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb8101e30 0x7fefb8102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.394+0000 7fefbd21d700 1 -- 172.21.15.47:0/934362481 >> 172.21.15.47:0/934362481 conn(0x7fefb80771f0 msgr2=0x7fefb8077600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:30.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.397+0000 7fefbd21d700 1 -- 172.21.15.47:0/934362481 shutdown_connections 2024-05-11T12:26:30.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.397+0000 7fefbd21d700 1 -- 172.21.15.47:0/934362481 wait complete. 2024-05-11T12:26:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.398+0000 7fefbd21d700 1 Processor -- start 2024-05-11T12:26:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.398+0000 7fefbd21d700 1 -- start start 2024-05-11T12:26:30.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.399+0000 7fefbd21d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb8101e30 0x7fefb8075090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.399+0000 7fefbd21d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 0x7fefb806d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.399+0000 7fefbd21d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 0x7fefb806d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.399+0000 7fefb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 0x7fefb806d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.399+0000 7fefb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 0x7fefb806d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40528/0 (socket says 172.21.15.47:40528) 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.399+0000 7fefb77fe700 1 -- 172.21.15.47:0/507106037 learned_addr learned my addr 172.21.15.47:0/507106037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.400+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8119d30 con 0x7fefb8109420 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.400+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8119bb0 con 0x7fefb8114160 2024-05-11T12:26:30.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.400+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefb8119eb0 con 0x7fefb8101e30 2024-05-11T12:26:30.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.401+0000 7fefb7fff700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb8101e30 0x7fefb8075090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.402+0000 7fefbca1c700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 0x7fefb806d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.403+0000 7fefb77fe700 1 -- 172.21.15.47:0/507106037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb8101e30 msgr2=0x7fefb8075090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.403+0000 7fefb77fe700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb8101e30 0x7fefb8075090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.403+0000 7fefb77fe700 1 -- 172.21.15.47:0/507106037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 msgr2=0x7fefb806d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.403+0000 7fefb77fe700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 0x7fefb806d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.403+0000 7fefb77fe700 1 -- 172.21.15.47:0/507106037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb81b4ed0 con 0x7fefb8109420 2024-05-11T12:26:30.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.403+0000 7fefb77fe700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 0x7fefb806d060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fefb0007fa0 tx=0x7fefb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:30.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.405+0000 7fefb57fa700 1 -- 172.21.15.47:0/507106037 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb001a070 con 0x7fefb8109420 2024-05-11T12:26:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.406+0000 7fefb57fa700 1 -- 172.21.15.47:0/507106037 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb000dda0 con 0x7fefb8109420 2024-05-11T12:26:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.406+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb81b50a0 con 0x7fefb8109420 2024-05-11T12:26:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.406+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb81b5550 con 0x7fefb8109420 2024-05-11T12:26:30.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.406+0000 7fefb57fa700 1 -- 172.21.15.47:0/507106037 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00146d0 con 0x7fefb8109420 2024-05-11T12:26:30.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.418+0000 7fefc8d15b80 1 Processor -- start 2024-05-11T12:26:30.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.422+0000 7fefc8d15b80 1 -- start start 2024-05-11T12:26:30.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.423+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710300 0x55b4567106e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.423+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710c20 0x55b456714890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.423+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456715210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.423+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b4566628e0 con 0x55b456710300 2024-05-11T12:26:30.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.423+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b456601840 con 0x55b456714dd0 2024-05-11T12:26:30.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.423+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b45665e8a0 con 0x55b456710c20 2024-05-11T12:26:30.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.423+0000 7fefb57fa700 1 -- 172.21.15.47:0/507106037 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fefb0014870 con 0x7fefb8109420 2024-05-11T12:26:30.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.424+0000 7fefbca1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710300 0x55b4567106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.424+0000 7fefbca1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710300 0x55b4567106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40544/0 (socket says 172.21.15.47:40544) 2024-05-11T12:26:30.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.424+0000 7fefbca1c700 1 -- 172.21.15.47:0/608320671 learned_addr learned my addr 172.21.15.47:0/608320671 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:30.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefb7fff700 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710c20 0x55b456714890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefbca1c700 1 -- 172.21.15.47:0/608320671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710c20 msgr2=0x55b456714890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:30.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefbca1c700 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710c20 0x55b456714890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefbca1c700 1 -- 172.21.15.47:0/608320671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 msgr2=0x55b456715210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:30.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefbca1c700 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456715210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefbca1c700 1 -- 172.21.15.47:0/608320671 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4565af950 con 0x55b456710300 2024-05-11T12:26:30.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefb7fff700 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710c20 0x55b456714890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:26:30.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefbca1c700 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710300 0x55b4567106e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fefac0098f0 tx=0x7fefac0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80d715bf4e7922a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:30.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.425+0000 7fefb57fa700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefa00b2a80 0x7fefa00b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.426+0000 7fef9e7fc700 1 -- 172.21.15.47:0/608320671 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac026070 con 0x55b456710300 2024-05-11T12:26:30.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.426+0000 7fef9e7fc700 1 -- 172.21.15.47:0/608320671 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac0176e0 con 0x55b456710300 2024-05-11T12:26:30.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.427+0000 7fefb7fff700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefa00b2a80 0x7fefa00b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.427+0000 7fefb57fa700 1 -- 172.21.15.47:0/507106037 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fefb00e19b0 con 0x7fefb8109420 2024-05-11T12:26:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.427+0000 7fefc8d15b80 1 -- 172.21.15.47:0/608320671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710300 msgr2=0x55b4567106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.427+0000 7fef9e7fc700 1 -- 172.21.15.47:0/608320671 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac026070 con 0x55b456710300 2024-05-11T12:26:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.427+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710300 0x55b4567106e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fefac0098f0 tx=0x7fefac0094e0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.428+0000 7fefc8d15b80 1 -- 172.21.15.47:0/608320671 shutdown_connections 2024-05-11T12:26:30.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.428+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456715210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.428+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710300 0x55b4567106e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.428+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/608320671 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710c20 0x55b456714890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.428+0000 7fefc8d15b80 1 -- 172.21.15.47:0/608320671 >> 172.21.15.47:0/608320671 conn(0x55b45663cee0 msgr2=0x55b45670ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:30.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.428+0000 7fefc8d15b80 1 -- 172.21.15.47:0/608320671 shutdown_connections 2024-05-11T12:26:30.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.430+0000 7fefc8d15b80 1 -- 172.21.15.47:0/608320671 wait complete. 2024-05-11T12:26:30.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.431+0000 7fefc8d15b80 1 -- start start 2024-05-11T12:26:30.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.431+0000 7fefc8d15b80 1 Processor -- start 2024-05-11T12:26:30.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.433+0000 7fefb7fff700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefa00b2a80 0x7fefa00b4f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55b4567111f0 tx=0x7fefa800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:30.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.433+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b45671c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.434+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456719530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.434+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456719a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.434+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b45671d020 con 0x55b456710c20 2024-05-11T12:26:30.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.434+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b4565c8cb0 con 0x55b456714dd0 2024-05-11T12:26:30.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.434+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b45671d1a0 con 0x55b456710300 2024-05-11T12:26:30.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.434+0000 7fefc8d15b80 10 client.0 _ll_register_callbacks cb 0x55b4565fa9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:30.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.435+0000 7fefb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456719530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.435+0000 7fefb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456719530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40558/0 (socket says 172.21.15.47:40558) 2024-05-11T12:26:30.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.435+0000 7fefb77fe700 1 -- 172.21.15.47:0/161537754 learned_addr learned my addr 172.21.15.47:0/161537754 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:30.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.436+0000 7fefb77fe700 1 -- 172.21.15.47:0/161537754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 msgr2=0x55b45671c190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:30.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.436+0000 7fefb77fe700 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b45671c190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.436+0000 7fefb77fe700 1 -- 172.21.15.47:0/161537754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 msgr2=0x55b456719a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:30.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.436+0000 7fefb77fe700 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456719a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.436+0000 7fefb77fe700 1 -- 172.21.15.47:0/161537754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4565fe710 con 0x55b456710c20 2024-05-11T12:26:30.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.438+0000 7fefb77fe700 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456719530 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fefb00a6110 tx=0x7fefb00a88a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27abde996a9790dd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:30.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.439+0000 7fef977fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:26:30.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.442+0000 7fef977fe700 1 -- 172.21.15.47:0/161537754 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00a95a0 con 0x55b456710c20 2024-05-11T12:26:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.442+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefac0176e0 con 0x55b456710c20 2024-05-11T12:26:30.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.442+0000 7fef977fe700 1 -- 172.21.15.47:0/161537754 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fefb00a5490 con 0x55b456710c20 2024-05-11T12:26:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.443+0000 7fef977fe700 1 -- 172.21.15.47:0/161537754 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fefb00c7030 con 0x55b456710c20 2024-05-11T12:26:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.444+0000 7fefc8d15b80 10 client.15324 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:30.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.444+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b4567277f0 con 0x55b456710c20 2024-05-11T12:26:30.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fef977fe700 1 -- 172.21.15.47:0/161537754 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefac0176e0 con 0x55b456710c20 2024-05-11T12:26:30.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fef977fe700 1 -- 172.21.15.47:0/161537754 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fefb00bf070 con 0x55b456710c20 2024-05-11T12:26:30.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fefc8d15b80 10 client.15324 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:30.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fefc8d15b80 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:30.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fefc8d15b80 20 client.15324 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:30.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b45672a230 0x55b45672c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fefc8d15b80 4 client.15324 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:26:30.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.445+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4566702c0 con 0x55b45672a230 2024-05-11T12:26:30.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.447+0000 7fefb7fff700 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b45672a230 0x55b45672c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.451+0000 7fefb7fff700 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b45672a230 0x55b45672c670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64caacca59030005 server_cookie=4670b20807aedd2a in_seq=0 out_seq=0 2024-05-11T12:26:30.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.452+0000 7fef977fe700 10 client.15324 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:30.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.458+0000 7fef977fe700 1 -- 172.21.15.47:0/161537754 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b4566702c0 con 0x55b45672a230 2024-05-11T12:26:30.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.458+0000 7fef977fe700 10 client.15324 handle_command_reply: tid=0 2024-05-11T12:26:30.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.458+0000 7fefc8d15b80 1 client.15324 shutdown 2024-05-11T12:26:30.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 msgr2=0x55b456719530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456719530 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fefb00a6110 tx=0x7fefb00a88a0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 shutdown_connections 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b45672a230 0x55b45672c670 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456719a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b45671c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/161537754 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456719530 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.459+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 >> 172.21.15.47:0/161537754 conn(0x55b45663cee0 msgr2=0x55b45670f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.460+0000 7fef977fe700 0 client.15324 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.460+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 shutdown_connections 2024-05-11T12:26:30.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.460+0000 7fefc8d15b80 1 -- 172.21.15.47:0/161537754 wait complete. 2024-05-11T12:26:30.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.461+0000 7fefc8d15b80 20 client.15324 trim_cache size 0 max 16384 2024-05-11T12:26:30.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.524+0000 7fefc8d15b80 1 Processor -- start 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.524+0000 7fefc8d15b80 1 -- start start 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.524+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456726190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.525+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456718e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.525+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b4567161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.525+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fefa400b830 con 0x55b456710c20 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.525+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefa400e0b0 con 0x55b456714dd0 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.525+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefa400bea0 con 0x55b456710300 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456718e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456718e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40564/0 (socket says 172.21.15.47:40564) 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 -- 172.21.15.47:0/1793069023 learned_addr learned my addr 172.21.15.47:0/1793069023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb7fff700 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456726190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 -- 172.21.15.47:0/1793069023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 msgr2=0x55b456726190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:30.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456726190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 -- 172.21.15.47:0/1793069023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 msgr2=0x55b4567161c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b4567161c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.526+0000 7fefb77fe700 1 -- 172.21.15.47:0/1793069023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b45671c970 con 0x55b456710c20 2024-05-11T12:26:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.527+0000 7fefb77fe700 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456718e50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fefb00a4f20 tx=0x7fefb0006070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d41efd6f2e91d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.528+0000 7fef9dffb700 1 -- 172.21.15.47:0/1793069023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb00a95a0 con 0x55b456710c20 2024-05-11T12:26:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.528+0000 7fef9dffb700 1 -- 172.21.15.47:0/1793069023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb00a8ea0 con 0x55b456710c20 2024-05-11T12:26:30.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.530+0000 7fefc8d15b80 1 -- 172.21.15.47:0/1793069023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 msgr2=0x55b456718e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.530+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456718e50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fefb00a4f20 tx=0x7fefb0006070 comp rx=0 tx=0).stop 2024-05-11T12:26:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.531+0000 7fefc8d15b80 1 -- 172.21.15.47:0/1793069023 shutdown_connections 2024-05-11T12:26:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.531+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b4567161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.531+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456726190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.531+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/1793069023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456710c20 0x55b456718e50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.531+0000 7fefc8d15b80 1 -- 172.21.15.47:0/1793069023 >> 172.21.15.47:0/1793069023 conn(0x55b45663cee0 msgr2=0x55b45670f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:30.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.531+0000 7fefc8d15b80 1 -- 172.21.15.47:0/1793069023 shutdown_connections 2024-05-11T12:26:30.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.531+0000 7fefc8d15b80 1 -- 172.21.15.47:0/1793069023 wait complete. 2024-05-11T12:26:30.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.532+0000 7fefc8d15b80 1 -- start start 2024-05-11T12:26:30.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.532+0000 7fefc8d15b80 1 Processor -- start 2024-05-11T12:26:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.535+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456731b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.536+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456732090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.536+0000 7fefc8d15b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456732620 0x55b456793af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.536+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fefa400c570 con 0x55b456732620 2024-05-11T12:26:30.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.536+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefa400d9e0 con 0x55b456714dd0 2024-05-11T12:26:30.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.536+0000 7fefc8d15b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefa400cc40 con 0x55b456710300 2024-05-11T12:26:30.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.536+0000 7fefc8d15b80 10 client.0 _ll_register_callbacks cb 0x55b4565fa9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:30.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.537+0000 7fefbca1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456732620 0x55b456793af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.537+0000 7fefbca1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456732620 0x55b456793af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40580/0 (socket says 172.21.15.47:40580) 2024-05-11T12:26:30.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.537+0000 7fefbca1c700 1 -- 172.21.15.47:0/710039277 learned_addr learned my addr 172.21.15.47:0/710039277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:30.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.539+0000 7fefb7fff700 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456731b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.540+0000 7fefbca1c700 1 -- 172.21.15.47:0/710039277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 msgr2=0x55b456731b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:30.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.540+0000 7fefbca1c700 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456731b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.540+0000 7fefbca1c700 1 -- 172.21.15.47:0/710039277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 msgr2=0x55b456732090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:30.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.540+0000 7fefbca1c700 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456732090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.540+0000 7fefbca1c700 1 -- 172.21.15.47:0/710039277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4567075e0 con 0x55b456732620 2024-05-11T12:26:30.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.542+0000 7fef977fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:26:30.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.542+0000 7fefbca1c700 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456732620 0x55b456793af0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac017df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20647026426c8182 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:30.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.544+0000 7fef977fe700 1 -- 172.21.15.47:0/710039277 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefac02f060 con 0x55b456732620 2024-05-11T12:26:30.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.544+0000 7fef977fe700 1 -- 172.21.15.47:0/710039277 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fefac005230 con 0x55b456732620 2024-05-11T12:26:30.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.544+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b45666a360 con 0x55b456732620 2024-05-11T12:26:30.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.545+0000 7fef977fe700 1 -- 172.21.15.47:0/710039277 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fefac026040 con 0x55b456732620 2024-05-11T12:26:30.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.545+0000 7fefc8d15b80 10 client.15330 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:30.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.545+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b4567a6460 con 0x55b456732620 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.547+0000 7fef977fe700 1 -- 172.21.15.47:0/710039277 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b45666a360 con 0x55b456732620 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.547+0000 7fef977fe700 1 -- 172.21.15.47:0/710039277 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fefac029070 con 0x55b456732620 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.548+0000 7fefc8d15b80 10 client.15330 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.548+0000 7fefc8d15b80 10 client.15330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.548+0000 7fefc8d15b80 20 client.15330 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.548+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b4567a6690 0x55b4567a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.548+0000 7fefc8d15b80 4 client.15330 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:26:30.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.548+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4564e22d0 con 0x55b4567a6690 2024-05-11T12:26:30.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.551+0000 7fefb7fff700 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b4567a6690 0x55b4567a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.572+0000 7fef977fe700 10 client.15330 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:30.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.572+0000 7fefb7fff700 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b4567a6690 0x55b4567a8a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3e02cdf95da0d0 server_cookie=67853c3c1a57cfc6 in_seq=0 out_seq=0 2024-05-11T12:26:30.707 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:26:30.707 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:26:30.707 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "1475b6f6-0fa7-4edd-b5bd-bb808b75e77a", 2024-05-11T12:26:30.707 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:26:30.707 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:26:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.697+0000 7fef977fe700 1 -- 172.21.15.47:0/710039277 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4564e22d0 con 0x55b4567a6690 2024-05-11T12:26:30.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.697+0000 7fef977fe700 10 client.15330 handle_command_reply: tid=0 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.697+0000 7fefc8d15b80 1 client.15330 shutdown 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.697+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456732620 msgr2=0x55b456793af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.697+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456732620 0x55b456793af0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac017df0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 shutdown_connections 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b4567a6690 0x55b4567a8a80 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b456732620 0x55b456793af0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b456710300 0x55b456731b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 --2- 172.21.15.47:0/710039277 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b456714dd0 0x55b456732090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 >> 172.21.15.47:0/710039277 conn(0x55b45663cee0 msgr2=0x55b45670f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fef977fe700 0 client.15330 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 shutdown_connections 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.698+0000 7fefc8d15b80 1 -- 172.21.15.47:0/710039277 wait complete. 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.699+0000 7fefc8d15b80 20 client.15330 trim_cache size 0 max 16384 2024-05-11T12:26:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.700+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefa00b2a80 msgr2=0x7fefa00b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.700+0000 7fefbd21d700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefa00b2a80 0x7fefa00b4f40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55b4567111f0 tx=0x7fefa800b410 comp rx=0 tx=0).stop 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.700+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 msgr2=0x7fefb806d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.700+0000 7fefbd21d700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 0x7fefb806d060 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fefb0007fa0 tx=0x7fefb000d3b0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 shutdown_connections 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefa00b2a80 0x7fefa00b4f40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefb8114160 0x7fefb806d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefb8101e30 0x7fefb8075090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 --2- 172.21.15.47:0/507106037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefb8109420 0x7fefb806d060 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 >> 172.21.15.47:0/507106037 conn(0x7fefb80771f0 msgr2=0x7fefb80fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 shutdown_connections 2024-05-11T12:26:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:30.704+0000 7fefbd21d700 1 -- 172.21.15.47:0/507106037 wait complete. 2024-05-11T12:26:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:31 smithi064 ceph-mon[34537]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 754 MiB/s wr, 2.17k op/s 2024-05-11T12:26:31.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:31 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:26:31.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:31 smithi050 ceph-mon[34072]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 754 MiB/s wr, 2.17k op/s 2024-05-11T12:26:31.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:31 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:26:31.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:31 smithi047 ceph-mon[30375]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 754 MiB/s wr, 2.17k op/s 2024-05-11T12:26:31.509 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:31 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:26:32.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:32 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:26:32.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:32 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:26:32.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:32 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:26:32.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:32 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:26:32.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:32 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:26:32.388 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:32 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:26:32.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:32 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:26:32.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:32 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:26:32.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:32 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:26:32.479 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:26:33.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:33 smithi047 ceph-mon[30375]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 801 MiB/s wr, 2.29k op/s 2024-05-11T12:26:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:33 smithi064 ceph-mon[34537]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 801 MiB/s wr, 2.29k op/s 2024-05-11T12:26:33.439 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:26:33.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:33 smithi050 ceph-mon[34072]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 801 MiB/s wr, 2.29k op/s 2024-05-11T12:26:34.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:26:34.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:26:34.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:34 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:26:34.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:34 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:26:34.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:34 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:26:34.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:26:34.380 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:26:34.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:34 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:26:34.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:34 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:26:34.381 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:34 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:26:34.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:26:34.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:26:34.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:34 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:26:34.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:34 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:26:34.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:34 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:26:35.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:35 smithi050 ceph-mon[34072]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 888 MiB/s wr, 2.34k op/s 2024-05-11T12:26:35.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:35 smithi047 ceph-mon[30375]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 888 MiB/s wr, 2.34k op/s 2024-05-11T12:26:35.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:35 smithi064 ceph-mon[34537]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 888 MiB/s wr, 2.34k op/s 2024-05-11T12:26:36.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:36 smithi047 ceph-mon[30375]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 780 MiB/s wr, 2.00k op/s 2024-05-11T12:26:36.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:36 smithi064 ceph-mon[34537]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 780 MiB/s wr, 2.00k op/s 2024-05-11T12:26:36.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:36 smithi050 ceph-mon[34072]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 780 MiB/s wr, 2.00k op/s 2024-05-11T12:26:38.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:38 smithi047 ceph-mon[30375]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 898 MiB/s wr, 2.39k op/s 2024-05-11T12:26:39.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.025+0000 7f67d6ffd700 1 -- 172.21.15.47:0/1602897195 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0017400 con 0x7f67d8102fb0 2024-05-11T12:26:39.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.025+0000 7f67de075700 1 -- 172.21.15.47:0/1602897195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d8102fb0 msgr2=0x7f67d8107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:39.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.025+0000 7f67de075700 1 --2- 172.21.15.47:0/1602897195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d8102fb0 0x7f67d8107000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009920 tx=0x7f67c0009c30 comp rx=0 tx=0).stop 2024-05-11T12:26:39.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.027+0000 7f67de075700 1 -- 172.21.15.47:0/1602897195 shutdown_connections 2024-05-11T12:26:39.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.027+0000 7f67de075700 1 --2- 172.21.15.47:0/1602897195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d81076e0 0x7f67d810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.027+0000 7f67de075700 1 --2- 172.21.15.47:0/1602897195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d8102600 0x7f67d81029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.027+0000 7f67de075700 1 --2- 172.21.15.47:0/1602897195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d8102fb0 0x7f67d8107000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.027+0000 7f67de075700 1 -- 172.21.15.47:0/1602897195 >> 172.21.15.47:0/1602897195 conn(0x7f67d80fded0 msgr2=0x7f67d81002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:39.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.031+0000 7f67de075700 1 -- 172.21.15.47:0/1602897195 shutdown_connections 2024-05-11T12:26:39.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.031+0000 7f67de075700 1 -- 172.21.15.47:0/1602897195 wait complete. 2024-05-11T12:26:39.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.033+0000 7f67de075700 1 Processor -- start 2024-05-11T12:26:39.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.035+0000 7f67de075700 1 -- start start 2024-05-11T12:26:39.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.037+0000 7f67de075700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d8102600 0x7f67d8117a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.037+0000 7f67de075700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d8102fb0 0x7f67d8117fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.037+0000 7f67de075700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81076e0 0x7f67d8112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.037+0000 7f67de075700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81114e0 con 0x7f67d81076e0 2024-05-11T12:26:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.037+0000 7f67de075700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8111360 con 0x7f67d8102600 2024-05-11T12:26:39.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.037+0000 7f67de075700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8111660 con 0x7f67d8102fb0 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81076e0 0x7f67d8112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81076e0 0x7f67d8112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49636/0 (socket says 172.21.15.47:49636) 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 -- 172.21.15.47:0/299381070 learned_addr learned my addr 172.21.15.47:0/299381070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 -- 172.21.15.47:0/299381070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d8102fb0 msgr2=0x7f67d8117fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d8102fb0 0x7f67d8117fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 -- 172.21.15.47:0/299381070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d8102600 msgr2=0x7f67d8117a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d8102600 0x7f67d8117a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.038+0000 7f67d7fff700 1 -- 172.21.15.47:0/299381070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d8113500 con 0x7f67d81076e0 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.040+0000 7f67d7fff700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81076e0 0x7f67d8112c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e940 tx=0x7f67d000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.040+0000 7f67d57fa700 1 -- 172.21.15.47:0/299381070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d000cbd0 con 0x7f67d81076e0 2024-05-11T12:26:39.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.040+0000 7f67d57fa700 1 -- 172.21.15.47:0/299381070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d000cd70 con 0x7f67d81076e0 2024-05-11T12:26:39.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.041+0000 7f67d57fa700 1 -- 172.21.15.47:0/299381070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0010430 con 0x7f67d81076e0 2024-05-11T12:26:39.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.042+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81137f0 con 0x7f67d81076e0 2024-05-11T12:26:39.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.042+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d811d980 con 0x7f67d81076e0 2024-05-11T12:26:39.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.051+0000 7f67e890bb80 1 Processor -- start 2024-05-11T12:26:39.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.052+0000 7f67d57fa700 1 -- 172.21.15.47:0/299381070 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f67d0004710 con 0x7f67d81076e0 2024-05-11T12:26:39.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.054+0000 7f67d57fa700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f67c40b2830 0x7f67c40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67d57fa700 1 -- 172.21.15.47:0/299381070 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f67d00e24a0 con 0x7f67d81076e0 2024-05-11T12:26:39.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67e890bb80 1 -- start start 2024-05-11T12:26:39.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d0280 0x558ba93d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 0x558ba93d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d4d50 0x558ba93d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67e890bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558ba9322860 con 0x558ba93d0ba0 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67e890bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558ba92c17c0 con 0x558ba93d4d50 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.055+0000 7f67e890bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ba931e820 con 0x558ba93d0280 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.056+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d4d50 0x558ba93d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.056+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d4d50 0x558ba93d5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:51460/0 (socket says 172.21.15.47:51460) 2024-05-11T12:26:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.056+0000 7f67d7fff700 1 -- 172.21.15.47:0/4082751604 learned_addr learned my addr 172.21.15.47:0/4082751604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.056+0000 7f67ceffd700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f67c40b2830 0x7f67c40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.057+0000 7f67d7fff700 1 -- 172.21.15.47:0/4082751604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d0280 msgr2=0x558ba93d0660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.057+0000 7f67d7fff700 1 --2- 172.21.15.47:0/4082751604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d0280 0x558ba93d0660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.057+0000 7f67d7fff700 1 -- 172.21.15.47:0/4082751604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 msgr2=0x558ba93d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.057+0000 7f67d7fff700 1 --2- 172.21.15.47:0/4082751604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 0x558ba93d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.057+0000 7f67d7fff700 1 -- 172.21.15.47:0/4082751604 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba926f8d0 con 0x558ba93d4d50 2024-05-11T12:26:39.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.057+0000 7f67ceffd700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f67c40b2830 0x7f67c40b4cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67c0000c00 tx=0x7f67c0009fa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.057+0000 7f67d7fff700 1 --2- 172.21.15.47:0/4082751604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d4d50 0x558ba93d5190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009230 tx=0x7f67d00a9470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6657f45897a1276 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:39.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.058+0000 7f67cdffb700 1 -- 172.21.15.47:0/4082751604 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00a9500 con 0x558ba93d4d50 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.059+0000 7f67cdffb700 1 -- 172.21.15.47:0/4082751604 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d00aa0f0 con 0x558ba93d4d50 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.060+0000 7f67cdffb700 1 -- 172.21.15.47:0/4082751604 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00b2620 con 0x558ba93d4d50 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.060+0000 7f67e890bb80 1 -- 172.21.15.47:0/4082751604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d4d50 msgr2=0x558ba93d5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.060+0000 7f67e890bb80 1 --2- 172.21.15.47:0/4082751604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d4d50 0x558ba93d5190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009230 tx=0x7f67d00a9470 comp rx=0 tx=0).stop 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.061+0000 7f67e890bb80 1 -- 172.21.15.47:0/4082751604 shutdown_connections 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.061+0000 7f67e890bb80 1 --2- 172.21.15.47:0/4082751604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d4d50 0x558ba93d5190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.061+0000 7f67e890bb80 1 --2- 172.21.15.47:0/4082751604 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d0280 0x558ba93d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.061+0000 7f67e890bb80 1 --2- 172.21.15.47:0/4082751604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 0x558ba93d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.061+0000 7f67e890bb80 1 -- 172.21.15.47:0/4082751604 >> 172.21.15.47:0/4082751604 conn(0x558ba92fce60 msgr2=0x558ba93cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.062+0000 7f67e890bb80 1 -- 172.21.15.47:0/4082751604 shutdown_connections 2024-05-11T12:26:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.062+0000 7f67e890bb80 1 -- 172.21.15.47:0/4082751604 wait complete. 2024-05-11T12:26:39.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.062+0000 7f67e890bb80 1 -- start start 2024-05-11T12:26:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.064+0000 7f67e890bb80 1 Processor -- start 2024-05-11T12:26:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.066+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0280 0x558ba93dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.066+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 0x558ba93d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.066+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.067+0000 7f67e890bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558ba93dd000 con 0x558ba93d0ba0 2024-05-11T12:26:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.067+0000 7f67e890bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558ba9288c30 con 0x558ba93d0280 2024-05-11T12:26:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.067+0000 7f67e890bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558ba93dd180 con 0x558ba93d4d50 2024-05-11T12:26:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.067+0000 7f67e890bb80 10 client.0 _ll_register_callbacks cb 0x558ba92ba920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.069+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.069+0000 7f67d77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0280 0x558ba93dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.069+0000 7f67d77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0280 0x558ba93dc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:51476/0 (socket says 172.21.15.47:51476) 2024-05-11T12:26:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.069+0000 7f67d77fe700 1 -- 172.21.15.47:0/1296196215 learned_addr learned my addr 172.21.15.47:0/1296196215 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.072+0000 7f67ceffd700 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 0x558ba93d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.073+0000 7f67d77fe700 1 -- 172.21.15.47:0/1296196215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 msgr2=0x558ba93d9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.073+0000 7f67d77fe700 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93d9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.073+0000 7f67d77fe700 1 -- 172.21.15.47:0/1296196215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 msgr2=0x558ba93d95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:39.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.073+0000 7f67d77fe700 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 0x558ba93d95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.073+0000 7f67d77fe700 1 -- 172.21.15.47:0/1296196215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba92be690 con 0x558ba93d0280 2024-05-11T12:26:39.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.074+0000 7f67d77fe700 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0280 0x558ba93dc250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67c800ec00 tx=0x7f67c800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43dd80edf7025b02 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.074+0000 7f67b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:26:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.074+0000 7f67b6ffd700 1 -- 172.21.15.47:0/1296196215 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c80092e0 con 0x558ba93d0280 2024-05-11T12:26:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.074+0000 7f67b6ffd700 1 -- 172.21.15.47:0/1296196215 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f67c8004bb0 con 0x558ba93d0280 2024-05-11T12:26:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.075+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d00aa0f0 con 0x558ba93d0280 2024-05-11T12:26:39.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.075+0000 7f67b6ffd700 1 -- 172.21.15.47:0/1296196215 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f67c8004db0 con 0x558ba93d0280 2024-05-11T12:26:39.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.076+0000 7f67e890bb80 10 client.24953 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:39.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.076+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558ba93e78d0 con 0x558ba93d0280 2024-05-11T12:26:39.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67b6ffd700 1 -- 172.21.15.47:0/1296196215 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d00aa0f0 con 0x558ba93d0280 2024-05-11T12:26:39.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67b6ffd700 1 -- 172.21.15.47:0/1296196215 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f67c80078d0 con 0x558ba93d0280 2024-05-11T12:26:39.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67e890bb80 10 client.24953 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:39.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67e890bb80 10 client.24953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:39.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67e890bb80 20 client.24953 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:39.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67e890bb80 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba93ea2c0 0x558ba93ec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67e890bb80 4 client.24953 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:26:39.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.077+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ba9330240 con 0x558ba93ea2c0 2024-05-11T12:26:39.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.081+0000 7f67ceffd700 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba93ea2c0 0x558ba93ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.083+0000 7f67ceffd700 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba93ea2c0 0x558ba93ec760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d3cedf0619b9ce server_cookie=e8d6c2a233b0e9cc in_seq=0 out_seq=0 2024-05-11T12:26:39.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.083+0000 7f67b6ffd700 10 client.24953 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.086+0000 7f67b6ffd700 1 -- 172.21.15.47:0/1296196215 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558ba9330240 con 0x558ba93ea2c0 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.086+0000 7f67b6ffd700 10 client.24953 handle_command_reply: tid=0 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.086+0000 7f67e890bb80 1 client.24953 shutdown 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.086+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0280 msgr2=0x558ba93dc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.086+0000 7f67e890bb80 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0280 0x558ba93dc250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f67c800ec00 tx=0x7f67c800ef10 comp rx=0 tx=0).stop 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 shutdown_connections 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba93ea2c0 0x558ba93ec760 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93d9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0280 0x558ba93dc250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 --2- 172.21.15.47:0/1296196215 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0ba0 0x558ba93d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 >> 172.21.15.47:0/1296196215 conn(0x558ba92fce60 msgr2=0x558ba93cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67b6ffd700 0 client.24953 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 shutdown_connections 2024-05-11T12:26:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 1 -- 172.21.15.47:0/1296196215 wait complete. 2024-05-11T12:26:39.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.087+0000 7f67e890bb80 20 client.24953 trim_cache size 0 max 16384 2024-05-11T12:26:39.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:38 smithi064 ceph-mon[34537]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 898 MiB/s wr, 2.39k op/s 2024-05-11T12:26:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.136+0000 7f67e890bb80 1 Processor -- start 2024-05-11T12:26:39.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.136+0000 7f67e890bb80 1 -- start start 2024-05-11T12:26:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.137+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0280 0x558ba93d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.137+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.138+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93dc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.138+0000 7f67e890bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81bab20 con 0x558ba93d0280 2024-05-11T12:26:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.138+0000 7f67e890bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81bd380 con 0x558ba93d0ba0 2024-05-11T12:26:39.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.138+0000 7f67e890bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81bb170 con 0x558ba93d4d50 2024-05-11T12:26:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.139+0000 7f67d77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0280 0x558ba93d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.139+0000 7f67d77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0280 0x558ba93d3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49668/0 (socket says 172.21.15.47:49668) 2024-05-11T12:26:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.139+0000 7f67d77fe700 1 -- 172.21.15.47:0/2425423073 learned_addr learned my addr 172.21.15.47:0/2425423073 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.139+0000 7f67ceffd700 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.139+0000 7f67d7fff700 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93dc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.141+0000 7f67d77fe700 1 -- 172.21.15.47:0/2425423073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 msgr2=0x558ba93dc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.141+0000 7f67d77fe700 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93dc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.141+0000 7f67d77fe700 1 -- 172.21.15.47:0/2425423073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 msgr2=0x558ba93dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.141+0000 7f67d77fe700 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.142+0000 7f67d77fe700 1 -- 172.21.15.47:0/2425423073 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ba93ea0a0 con 0x558ba93d0280 2024-05-11T12:26:39.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.143+0000 7f67d77fe700 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0280 0x558ba93d3e20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c4d0 tx=0x7f67c8013630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95532102b41b591 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.154+0000 7f67cd7fa700 1 -- 172.21.15.47:0/2425423073 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c80092e0 con 0x558ba93d0280 2024-05-11T12:26:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.154+0000 7f67cd7fa700 1 -- 172.21.15.47:0/2425423073 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c80075c0 con 0x558ba93d0280 2024-05-11T12:26:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.157+0000 7f67e890bb80 1 -- 172.21.15.47:0/2425423073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0280 msgr2=0x558ba93d3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:39.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.157+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0280 0x558ba93d3e20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c4d0 tx=0x7f67c8013630 comp rx=0 tx=0).stop 2024-05-11T12:26:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.159+0000 7f67e890bb80 1 -- 172.21.15.47:0/2425423073 shutdown_connections 2024-05-11T12:26:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.159+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93dc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.159+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93d0280 0x558ba93d3e20 secure :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c4d0 tx=0x7f67c8013630 comp rx=0 tx=0).stop 2024-05-11T12:26:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.159+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2425423073 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.159+0000 7f67e890bb80 1 -- 172.21.15.47:0/2425423073 >> 172.21.15.47:0/2425423073 conn(0x558ba92fce60 msgr2=0x558ba93d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:39.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.167+0000 7f67e890bb80 1 -- 172.21.15.47:0/2425423073 shutdown_connections 2024-05-11T12:26:39.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.167+0000 7f67e890bb80 1 -- 172.21.15.47:0/2425423073 wait complete. 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.168+0000 7f67e890bb80 1 -- start start 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 1 Processor -- start 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93e0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93e0a70 0x558ba9453c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81bb840 con 0x558ba93e0a70 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81bccb0 con 0x558ba93d0ba0 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81bbf10 con 0x558ba93d4d50 2024-05-11T12:26:39.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.169+0000 7f67e890bb80 10 client.0 _ll_register_callbacks cb 0x558ba92ba920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.172+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93e0a70 0x558ba9453c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.172+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93e0a70 0x558ba9453c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49678/0 (socket says 172.21.15.47:49678) 2024-05-11T12:26:39.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.172+0000 7f67d7fff700 1 -- 172.21.15.47:0/2671889149 learned_addr learned my addr 172.21.15.47:0/2671889149 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:39.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.174+0000 7f67d77fe700 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.174+0000 7f67ceffd700 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.175+0000 7f67d7fff700 1 -- 172.21.15.47:0/2671889149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 msgr2=0x558ba93e0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:39.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.175+0000 7f67d7fff700 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93e0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.175+0000 7f67d7fff700 1 -- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 msgr2=0x558ba93dfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:39.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.175+0000 7f67d7fff700 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.175+0000 7f67d7fff700 1 -- 172.21.15.47:0/2671889149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ba93c7560 con 0x558ba93e0a70 2024-05-11T12:26:39.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.181+0000 7f67d7fff700 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93e0a70 0x558ba9453c30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f67d00a9980 tx=0x7f67d00092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e3507f23ea02bb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:39.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.181+0000 7f67b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:26:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.190+0000 7f67b6ffd700 1 -- 172.21.15.47:0/2671889149 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00a99f0 con 0x558ba93e0a70 2024-05-11T12:26:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.190+0000 7f67b6ffd700 1 -- 172.21.15.47:0/2671889149 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f67d00cc2d0 con 0x558ba93e0a70 2024-05-11T12:26:39.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.191+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ba932a2e0 con 0x558ba93e0a70 2024-05-11T12:26:39.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.192+0000 7f67b6ffd700 1 -- 172.21.15.47:0/2671889149 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f67d00b2850 con 0x558ba93e0a70 2024-05-11T12:26:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.193+0000 7f67e890bb80 10 client.15345 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:39.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.193+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558ba94662b0 con 0x558ba93e0a70 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.195+0000 7f67b6ffd700 1 -- 172.21.15.47:0/2671889149 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558ba932a2e0 con 0x558ba93e0a70 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.196+0000 7f67b6ffd700 1 -- 172.21.15.47:0/2671889149 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f67d00a5360 con 0x558ba93e0a70 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.196+0000 7f67e890bb80 10 client.15345 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.196+0000 7f67e890bb80 10 client.15345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.196+0000 7f67e890bb80 20 client.15345 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.196+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba94668b0 0x558ba9468cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.196+0000 7f67e890bb80 4 client.15345 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:26:39.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.196+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ba91a2250 con 0x558ba94668b0 2024-05-11T12:26:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:38 smithi050 ceph-mon[34072]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 898 MiB/s wr, 2.39k op/s 2024-05-11T12:26:39.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.201+0000 7f67d77fe700 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba94668b0 0x558ba9468cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:39.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.204+0000 7f67d77fe700 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba94668b0 0x558ba9468cd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f8e9102136a943f server_cookie=ce7f39694b2af94d in_seq=0 out_seq=0 2024-05-11T12:26:39.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.204+0000 7f67b6ffd700 10 client.15345 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:39.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.213+0000 7f67b6ffd700 1 -- 172.21.15.47:0/2671889149 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558ba91a2250 con 0x558ba94668b0 2024-05-11T12:26:39.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.213+0000 7f67b6ffd700 10 client.15345 handle_command_reply: tid=0 2024-05-11T12:26:39.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.213+0000 7f67e890bb80 1 client.15345 shutdown 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.219+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93e0a70 msgr2=0x558ba9453c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.219+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93e0a70 0x558ba9453c30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f67d00a9980 tx=0x7f67d00092f0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 shutdown_connections 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558ba94668b0 0x558ba9468cd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558ba93e0a70 0x558ba9453c30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558ba93d0ba0 0x558ba93dfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 --2- 172.21.15.47:0/2671889149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558ba93d4d50 0x558ba93e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 >> 172.21.15.47:0/2671889149 conn(0x558ba92fce60 msgr2=0x558ba93d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:39.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67b6ffd700 0 client.15345 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 shutdown_connections 2024-05-11T12:26:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 1 -- 172.21.15.47:0/2671889149 wait complete. 2024-05-11T12:26:39.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.220+0000 7f67e890bb80 20 client.15345 trim_cache size 0 max 16384 2024-05-11T12:26:39.226 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:26:39.227 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:26:39.227 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:26:39.227 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:26:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.230+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f67c40b2830 msgr2=0x7f67c40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:39.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.230+0000 7f67de075700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f67c40b2830 0x7f67c40b4cf0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67c0000c00 tx=0x7f67c0009fa0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.230+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81076e0 msgr2=0x7f67d8112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.230+0000 7f67de075700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81076e0 0x7f67d8112c00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e940 tx=0x7f67d000ed00 comp rx=0 tx=0).stop 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.233+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 shutdown_connections 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.233+0000 7f67de075700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f67d81076e0 0x7f67d8112c00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.233+0000 7f67de075700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f67c40b2830 0x7f67c40b4cf0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.233+0000 7f67de075700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f67d8102600 0x7f67d8117a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.233+0000 7f67de075700 1 --2- 172.21.15.47:0/299381070 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f67d8102fb0 0x7f67d8117fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.233+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 >> 172.21.15.47:0/299381070 conn(0x7f67d80fded0 msgr2=0x7f67d80ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.233+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 shutdown_connections 2024-05-11T12:26:39.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:39.234+0000 7f67de075700 1 -- 172.21.15.47:0/299381070 wait complete. 2024-05-11T12:26:41.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:40 smithi064 ceph-mon[34537]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 777 MiB/s wr, 2.02k op/s 2024-05-11T12:26:41.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:40 smithi050 ceph-mon[34072]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 777 MiB/s wr, 2.02k op/s 2024-05-11T12:26:41.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:40 smithi047 ceph-mon[30375]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 777 MiB/s wr, 2.02k op/s 2024-05-11T12:26:41.494 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:26:41.495 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:26:42.598 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:26:43.058 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:42 smithi047 ceph-mon[30375]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 791 MiB/s wr, 1.95k op/s 2024-05-11T12:26:43.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:42 smithi064 ceph-mon[34537]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 791 MiB/s wr, 1.95k op/s 2024-05-11T12:26:43.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:42 smithi050 ceph-mon[34072]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 791 MiB/s wr, 1.95k op/s 2024-05-11T12:26:45.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:44 smithi047 ceph-mon[30375]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 2.18k op/s 2024-05-11T12:26:45.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:44 smithi064 ceph-mon[34537]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 2.18k op/s 2024-05-11T12:26:45.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:44 smithi050 ceph-mon[34072]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 876 MiB/s wr, 2.18k op/s 2024-05-11T12:26:46.929 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:46 smithi047 ceph-mon[30375]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 761 MiB/s wr, 2.03k op/s 2024-05-11T12:26:47.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:46 smithi064 ceph-mon[34537]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 761 MiB/s wr, 2.03k op/s 2024-05-11T12:26:47.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:46 smithi050 ceph-mon[34072]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 761 MiB/s wr, 2.03k op/s 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.061+0000 7f04b3a11700 1 -- 172.21.15.47:0/3035716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac10a7b0 msgr2=0x7f04ac10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.061+0000 7f04b3a11700 1 --2- 172.21.15.47:0/3035716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac10a7b0 0x7f04ac10e7e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f04a80098f0 tx=0x7f04a8009c00 comp rx=0 tx=0).stop 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.062+0000 7f04b3a11700 1 -- 172.21.15.47:0/3035716755 shutdown_connections 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.062+0000 7f04b3a11700 1 --2- 172.21.15.47:0/3035716755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04ac079930 0x7f04ac07bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.062+0000 7f04b3a11700 1 --2- 172.21.15.47:0/3035716755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac10a7b0 0x7f04ac10e7e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.062+0000 7f04b3a11700 1 --2- 172.21.15.47:0/3035716755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04ac10ed20 0x7f04ac10f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.062+0000 7f04b3a11700 1 -- 172.21.15.47:0/3035716755 >> 172.21.15.47:0/3035716755 conn(0x7f04ac06e6c0 msgr2=0x7f04ac06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.062+0000 7f04b3a11700 1 -- 172.21.15.47:0/3035716755 shutdown_connections 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.062+0000 7f04b3a11700 1 -- 172.21.15.47:0/3035716755 wait complete. 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.065+0000 7f04b3a11700 1 Processor -- start 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.066+0000 7f04b3a11700 1 -- start start 2024-05-11T12:26:48.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.066+0000 7f04b3a11700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac079930 0x7f04ac085de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.066+0000 7f04b3a11700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04ac10ed20 0x7f04ac0781d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.066+0000 7f04b3a11700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04ac078710 0x7f04ac078b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.066+0000 7f04b3a11700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac07e760 con 0x7f04ac10ed20 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.066+0000 7f04b3a11700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac07e5e0 con 0x7f04ac079930 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.066+0000 7f04b3a11700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac07e8e0 con 0x7f04ac078710 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.071+0000 7f04b0fac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04ac10ed20 0x7f04ac0781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.071+0000 7f04b0fac700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04ac10ed20 0x7f04ac0781d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59982/0 (socket says 172.21.15.47:59982) 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.071+0000 7f04b0fac700 1 -- 172.21.15.47:0/2525787033 learned_addr learned my addr 172.21.15.47:0/2525787033 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.071+0000 7f04b1fae700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04ac078710 0x7f04ac078b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.071+0000 7f04b17ad700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac079930 0x7f04ac085de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.072+0000 7f04b17ad700 1 -- 172.21.15.47:0/2525787033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04ac078710 msgr2=0x7f04ac078b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.072+0000 7f04b17ad700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04ac078710 0x7f04ac078b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.072+0000 7f04b17ad700 1 -- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04ac10ed20 msgr2=0x7f04ac0781d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.072+0000 7f04b17ad700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04ac10ed20 0x7f04ac0781d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.072+0000 7f04b17ad700 1 -- 172.21.15.47:0/2525787033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac138aa0 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.073+0000 7f04b17ad700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac079930 0x7f04ac085de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009230 tx=0x7f04a800f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.073+0000 7f04a27fc700 1 -- 172.21.15.47:0/2525787033 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8004910 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.073+0000 7f04b3a11700 1 -- 172.21.15.47:0/2525787033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac138d30 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.073+0000 7f04b3a11700 1 -- 172.21.15.47:0/2525787033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac139260 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.074+0000 7f04a27fc700 1 -- 172.21.15.47:0/2525787033 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a8009350 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.074+0000 7f04a27fc700 1 -- 172.21.15.47:0/2525787033 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a8017ca0 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.079+0000 7f04a27fc700 1 -- 172.21.15.47:0/2525787033 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f04a800fb10 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.081+0000 7f04a27fc700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04980b29b0 0x7f04980b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.081+0000 7f04a27fc700 1 -- 172.21.15.47:0/2525787033 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f04a80e3660 con 0x7f04ac079930 2024-05-11T12:26:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.083+0000 7f04b0fac700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04980b29b0 0x7f04980b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.084+0000 7f04b3a11700 1 -- 172.21.15.47:0/2525787033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0490005330 con 0x7f04ac079930 2024-05-11T12:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.085+0000 7f04b0fac700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04980b29b0 0x7f04980b4e70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04ac075160 tx=0x7f04a400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.099+0000 7f04a27fc700 1 -- 172.21.15.47:0/2525787033 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04a8027060 con 0x7f04ac079930 2024-05-11T12:26:48.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.713+0000 7f04b3a11700 1 -- 172.21.15.47:0/2525787033 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0490005620 con 0x7f04ac079930 2024-05-11T12:26:48.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.715+0000 7f04a27fc700 1 -- 172.21.15.47:0/2525787033 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f04a8013030 con 0x7f04ac079930 2024-05-11T12:26:48.724 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:26:48.724 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:26:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.725+0000 7f0497fff700 1 -- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04980b29b0 msgr2=0x7f04980b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04980b29b0 0x7f04980b4e70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f04ac075160 tx=0x7f04a400b040 comp rx=0 tx=0).stop 2024-05-11T12:26:48.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 -- 172.21.15.47:0/2525787033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac079930 msgr2=0x7f04ac085de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac079930 0x7f04ac085de0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009230 tx=0x7f04a800f710 comp rx=0 tx=0).stop 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 -- 172.21.15.47:0/2525787033 shutdown_connections 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f04ac078710 0x7f04ac078b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f04980b29b0 0x7f04980b4e70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f04ac079930 0x7f04ac085de0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 --2- 172.21.15.47:0/2525787033 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04ac10ed20 0x7f04ac0781d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 -- 172.21.15.47:0/2525787033 >> 172.21.15.47:0/2525787033 conn(0x7f04ac06e6c0 msgr2=0x7f04ac0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 -- 172.21.15.47:0/2525787033 shutdown_connections 2024-05-11T12:26:48.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:48.726+0000 7f0497fff700 1 -- 172.21.15.47:0/2525787033 wait complete. 2024-05-11T12:26:48.735 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:26:49.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:48 smithi047 ceph-mon[30375]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 866 MiB/s wr, 2.16k op/s 2024-05-11T12:26:49.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:48 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2525787033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:26:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:48 smithi064 ceph-mon[34537]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 866 MiB/s wr, 2.16k op/s 2024-05-11T12:26:49.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:48 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2525787033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:26:49.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:48 smithi050 ceph-mon[34072]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 866 MiB/s wr, 2.16k op/s 2024-05-11T12:26:49.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:48 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2525787033' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:26:50.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:26:51.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:50 smithi064 ceph-mon[34537]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 773 MiB/s wr, 1.80k op/s 2024-05-11T12:26:51.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:50 smithi050 ceph-mon[34072]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 773 MiB/s wr, 1.80k op/s 2024-05-11T12:26:51.284 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:50 smithi047 ceph-mon[30375]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 773 MiB/s wr, 1.80k op/s 2024-05-11T12:26:51.830 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:26:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:52 smithi064 ceph-mon[34537]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.83k op/s 2024-05-11T12:26:53.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:52 smithi047 ceph-mon[30375]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.83k op/s 2024-05-11T12:26:53.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:52 smithi050 ceph-mon[34072]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.83k op/s 2024-05-11T12:26:55.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:54 smithi050 ceph-mon[34072]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 1.88k op/s 2024-05-11T12:26:55.280 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:54 smithi047 ceph-mon[30375]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 1.88k op/s 2024-05-11T12:26:55.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:54 smithi064 ceph-mon[34537]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 895 MiB/s wr, 1.88k op/s 2024-05-11T12:26:57.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:56 smithi050 ceph-mon[34072]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 780 MiB/s wr, 1.51k op/s 2024-05-11T12:26:57.266 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:56 smithi047 ceph-mon[30375]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 780 MiB/s wr, 1.51k op/s 2024-05-11T12:26:57.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:56 smithi064 ceph-mon[34537]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 780 MiB/s wr, 1.51k op/s 2024-05-11T12:26:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.592+0000 7f2785ffb700 1 -- 172.21.15.47:0/3370815767 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c003c00 con 0x7f2788118550 2024-05-11T12:26:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.599+0000 7f2787fff700 1 -- 172.21.15.47:0/3370815767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2788118550 msgr2=0x7f278811a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.599+0000 7f2787fff700 1 --2- 172.21.15.47:0/3370815767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2788118550 0x7f278811a940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f277c009990 tx=0x7f277c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.600+0000 7f2787fff700 1 -- 172.21.15.47:0/3370815767 shutdown_connections 2024-05-11T12:26:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.600+0000 7f2787fff700 1 --2- 172.21.15.47:0/3370815767 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2788118550 0x7f278811a940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.600+0000 7f2787fff700 1 --2- 172.21.15.47:0/3370815767 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f278810dc00 0x7f278810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.600+0000 7f2787fff700 1 --2- 172.21.15.47:0/3370815767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810aa90 0x7f278810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.600+0000 7f2787fff700 1 -- 172.21.15.47:0/3370815767 >> 172.21.15.47:0/3370815767 conn(0x7f278806e530 msgr2=0x7f278806e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.603+0000 7f2787fff700 1 -- 172.21.15.47:0/3370815767 shutdown_connections 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.603+0000 7f2787fff700 1 -- 172.21.15.47:0/3370815767 wait complete. 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.604+0000 7f2787fff700 1 Processor -- start 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.604+0000 7f2787fff700 1 -- start start 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.605+0000 7f2787fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f278810aa90 0x7f27881a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.605+0000 7f2787fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810dc00 0x7f27881a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.605+0000 7f2787fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2788118550 0x7f27881a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.605+0000 7f2787fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f278811e150 con 0x7f2788118550 2024-05-11T12:26:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.605+0000 7f2787fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f278811dfd0 con 0x7f278810dc00 2024-05-11T12:26:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.605+0000 7f2787fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f278811e2d0 con 0x7f278810aa90 2024-05-11T12:26:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.605+0000 7f27867fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810dc00 0x7f27881a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.606+0000 7f27867fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810dc00 0x7f27881a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:47400/0 (socket says 172.21.15.47:47400) 2024-05-11T12:26:57.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.606+0000 7f27867fc700 1 -- 172.21.15.47:0/226900002 learned_addr learned my addr 172.21.15.47:0/226900002 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:57.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.606+0000 7f2786ffd700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f278810aa90 0x7f27881a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.607+0000 7f27867fc700 1 -- 172.21.15.47:0/226900002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f278810aa90 msgr2=0x7f27881a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.607+0000 7f27867fc700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f278810aa90 0x7f27881a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.607+0000 7f27867fc700 1 -- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2788118550 msgr2=0x7f27881a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.607+0000 7f27867fc700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2788118550 0x7f27881a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.607+0000 7f27867fc700 1 -- 172.21.15.47:0/226900002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27881a9590 con 0x7f278810dc00 2024-05-11T12:26:57.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.608+0000 7f27867fc700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810dc00 0x7f27881a4ac0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f27800060b0 tx=0x7f278000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.609+0000 7f276ffff700 1 -- 172.21.15.47:0/226900002 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278001a070 con 0x7f278810dc00 2024-05-11T12:26:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.609+0000 7f276ffff700 1 -- 172.21.15.47:0/226900002 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278000ddd0 con 0x7f278810dc00 2024-05-11T12:26:57.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.609+0000 7f276ffff700 1 -- 172.21.15.47:0/226900002 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780014690 con 0x7f278810dc00 2024-05-11T12:26:57.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.609+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27881a9880 con 0x7f278810dc00 2024-05-11T12:26:57.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.609+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278810e100 con 0x7f278810dc00 2024-05-11T12:26:57.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.616+0000 7f2797bf4b80 1 Processor -- start 2024-05-11T12:26:57.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.616+0000 7f2797bf4b80 1 -- start start 2024-05-11T12:26:57.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.617+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279280 0x55aa3f279660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.617+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f27d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.617+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f27e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.617+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55aa3f1cb860 con 0x55aa3f27dd50 2024-05-11T12:26:57.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.617+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aa3f16a7c0 con 0x55aa3f279280 2024-05-11T12:26:57.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.617+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa3f1c7820 con 0x55aa3f279ba0 2024-05-11T12:26:57.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.621+0000 7f27867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f27e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.622+0000 7f27867fc700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f27e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34070/0 (socket says 172.21.15.47:34070) 2024-05-11T12:26:57.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.622+0000 7f27867fc700 1 -- 172.21.15.47:0/2344991587 learned_addr learned my addr 172.21.15.47:0/2344991587 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:57.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.622+0000 7f2786ffd700 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f27d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.622+0000 7f27877fe700 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279280 0x55aa3f279660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.623+0000 7f276ffff700 1 -- 172.21.15.47:0/226900002 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2780014830 con 0x7f278810dc00 2024-05-11T12:26:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.623+0000 7f2786ffd700 1 -- 172.21.15.47:0/2344991587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279280 msgr2=0x55aa3f279660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.623+0000 7f2786ffd700 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279280 0x55aa3f279660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.623+0000 7f2786ffd700 1 -- 172.21.15.47:0/2344991587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 msgr2=0x55aa3f27e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:57.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.623+0000 7f2786ffd700 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f27e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.623+0000 7f2786ffd700 1 -- 172.21.15.47:0/2344991587 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa3f1188d0 con 0x55aa3f279ba0 2024-05-11T12:26:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.623+0000 7f2786ffd700 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f27d810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a20 tx=0x7f2778009d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa21faee313e6727 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.624+0000 7f276ffff700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f27700b29c0 0x7f27700b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.625+0000 7f276ffff700 1 -- 172.21.15.47:0/226900002 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f27800e3c90 con 0x7f278810dc00 2024-05-11T12:26:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.625+0000 7f276d7fa700 1 -- 172.21.15.47:0/2344991587 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778017070 con 0x55aa3f279ba0 2024-05-11T12:26:57.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.625+0000 7f276d7fa700 1 -- 172.21.15.47:0/2344991587 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277800ed80 con 0x55aa3f279ba0 2024-05-11T12:26:57.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.627+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2344991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 msgr2=0x55aa3f27d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.627+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f27d810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a20 tx=0x7f2778009d30 comp rx=0 tx=0).stop 2024-05-11T12:26:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.627+0000 7f27877fe700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f27700b29c0 0x7f27700b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.628+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2344991587 shutdown_connections 2024-05-11T12:26:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.628+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f27e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.628+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279280 0x55aa3f279660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.628+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2344991587 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f27d810 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.628+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2344991587 >> 172.21.15.47:0/2344991587 conn(0x55aa3f1a5e60 msgr2=0x55aa3f273cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:57.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.628+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2344991587 shutdown_connections 2024-05-11T12:26:57.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.628+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2344991587 wait complete. 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.629+0000 7f27877fe700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f27700b29c0 0x7f27700b4e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f277c006010 tx=0x7f277c0096f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.629+0000 7f2797bf4b80 1 -- start start 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.629+0000 7f2797bf4b80 1 Processor -- start 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.630+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f285230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.630+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f2825d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.630+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f27dd50 0x55aa3f282b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.630+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55aa3f296fb0 con 0x55aa3f279280 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.630+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55aa3f131c30 con 0x55aa3f279ba0 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.630+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55aa3f297130 con 0x55aa3f27dd50 2024-05-11T12:26:57.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.630+0000 7f2797bf4b80 10 client.0 _ll_register_callbacks cb 0x55aa3f163920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.635+0000 7f27877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f27dd50 0x55aa3f282b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.635+0000 7f27877fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f27dd50 0x55aa3f282b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53272/0 (socket says 172.21.15.47:53272) 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.635+0000 7f27877fe700 1 -- 172.21.15.47:0/2375429081 learned_addr learned my addr 172.21.15.47:0/2375429081 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.635+0000 7f27877fe700 1 -- 172.21.15.47:0/2375429081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 msgr2=0x55aa3f2825d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.635+0000 7f27877fe700 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f2825d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.635+0000 7f27877fe700 1 -- 172.21.15.47:0/2375429081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 msgr2=0x55aa3f285230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.635+0000 7f27877fe700 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f285230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.636+0000 7f27877fe700 1 -- 172.21.15.47:0/2375429081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa3f167690 con 0x55aa3f27dd50 2024-05-11T12:26:57.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.636+0000 7f27877fe700 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f27dd50 0x55aa3f282b10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f277c00bf90 tx=0x7f277c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f5e91f1764127e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.644+0000 7f276aff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:26:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.644+0000 7f276aff5700 1 -- 172.21.15.47:0/2375429081 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c035060 con 0x55aa3f27dd50 2024-05-11T12:26:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.644+0000 7f276aff5700 1 -- 172.21.15.47:0/2375429081 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f277c025a40 con 0x55aa3f27dd50 2024-05-11T12:26:57.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.644+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa3f1d9240 con 0x55aa3f27dd50 2024-05-11T12:26:57.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.644+0000 7f276aff5700 1 -- 172.21.15.47:0/2375429081 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f277c025c40 con 0x55aa3f27dd50 2024-05-11T12:26:57.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.645+0000 7f2797bf4b80 10 client.25009 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:57.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.645+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55aa3f2a1a00 con 0x55aa3f27dd50 2024-05-11T12:26:57.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.646+0000 7f276aff5700 1 -- 172.21.15.47:0/2375429081 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55aa3f1d9240 con 0x55aa3f27dd50 2024-05-11T12:26:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.647+0000 7f276aff5700 1 -- 172.21.15.47:0/2375429081 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f277c030460 con 0x55aa3f27dd50 2024-05-11T12:26:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.648+0000 7f2797bf4b80 10 client.25009 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.648+0000 7f2797bf4b80 10 client.25009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.648+0000 7f2797bf4b80 20 client.25009 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.648+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f2a43f0 0x55aa3f2a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.648+0000 7f2797bf4b80 4 client.25009 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:26:57.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.648+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa3f1d32e0 con 0x55aa3f2a43f0 2024-05-11T12:26:57.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.651+0000 7f2786ffd700 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f2a43f0 0x55aa3f2a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.670+0000 7f276aff5700 10 client.25009 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:57.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.670+0000 7f2786ffd700 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f2a43f0 0x55aa3f2a68b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cd6136321cf4d8f server_cookie=94d372db38e7ea27 in_seq=0 out_seq=0 2024-05-11T12:26:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.672+0000 7f276aff5700 1 -- 172.21.15.47:0/2375429081 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aa3f1d32e0 con 0x55aa3f2a43f0 2024-05-11T12:26:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.672+0000 7f276aff5700 10 client.25009 handle_command_reply: tid=0 2024-05-11T12:26:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.672+0000 7f2797bf4b80 1 client.25009 shutdown 2024-05-11T12:26:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f27dd50 msgr2=0x55aa3f282b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f27dd50 0x55aa3f282b10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f277c00bf90 tx=0x7f277c009310 comp rx=0 tx=0).stop 2024-05-11T12:26:57.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 shutdown_connections 2024-05-11T12:26:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f2a43f0 0x55aa3f2a68b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f27dd50 0x55aa3f282b10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f285230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/2375429081 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f2825d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 >> 172.21.15.47:0/2375429081 conn(0x55aa3f1a5e60 msgr2=0x55aa3f278090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:57.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.685+0000 7f276aff5700 0 client.25009 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:57.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.694+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 shutdown_connections 2024-05-11T12:26:57.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.694+0000 7f2797bf4b80 1 -- 172.21.15.47:0/2375429081 wait complete. 2024-05-11T12:26:57.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.694+0000 7f2797bf4b80 20 client.25009 trim_cache size 0 max 16384 2024-05-11T12:26:57.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.733+0000 7f2797bf4b80 1 Processor -- start 2024-05-11T12:26:57.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.733+0000 7f2797bf4b80 1 -- start start 2024-05-11T12:26:57.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.734+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279280 0x55aa3f2a0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.734+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f280ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f281420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f27877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f281420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f27877fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f281420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34074/0 (socket says 172.21.15.47:34074) 2024-05-11T12:26:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f2786ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279280 0x55aa3f2a0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f2786ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279280 0x55aa3f2a0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53288/0 (socket says 172.21.15.47:53288) 2024-05-11T12:26:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f27867fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f280ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f27867fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f280ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:47414/0 (socket says 172.21.15.47:47414) 2024-05-11T12:26:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.735+0000 7f27877fe700 1 -- 172.21.15.47:0/3268431834 learned_addr learned my addr 172.21.15.47:0/3268431834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.736+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f277400b830 con 0x55aa3f27dd50 2024-05-11T12:26:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.736+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f277400e0b0 con 0x55aa3f279ba0 2024-05-11T12:26:57.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.736+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f277400bea0 con 0x55aa3f279280 2024-05-11T12:26:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.737+0000 7f27877fe700 1 -- 172.21.15.47:0/3268431834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279280 msgr2=0x55aa3f2a0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.737+0000 7f27877fe700 1 --2- 172.21.15.47:0/3268431834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279280 0x55aa3f2a0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.737+0000 7f27877fe700 1 -- 172.21.15.47:0/3268431834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 msgr2=0x55aa3f280ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:26:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.737+0000 7f27877fe700 1 --2- 172.21.15.47:0/3268431834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f280ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.737+0000 7f27877fe700 1 -- 172.21.15.47:0/3268431834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa3f280b60 con 0x55aa3f27dd50 2024-05-11T12:26:57.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.738+0000 7f27877fe700 1 --2- 172.21.15.47:0/3268431834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f281420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b6e0 tx=0x7f277c025bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3d6745959a1173 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.739+0000 7f276cff9700 1 -- 172.21.15.47:0/3268431834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c035070 con 0x55aa3f27dd50 2024-05-11T12:26:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.739+0000 7f276cff9700 1 -- 172.21.15.47:0/3268431834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c030420 con 0x55aa3f27dd50 2024-05-11T12:26:57.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.739+0000 7f276cff9700 1 -- 172.21.15.47:0/3268431834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c03f080 con 0x55aa3f27dd50 2024-05-11T12:26:57.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.741+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 msgr2=0x55aa3f281420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.741+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/3268431834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f281420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b6e0 tx=0x7f277c025bd0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.741+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 shutdown_connections 2024-05-11T12:26:57.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.741+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/3268431834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f27dd50 0x55aa3f281420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.741+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/3268431834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279280 0x55aa3f2a0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.741+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/3268431834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f279ba0 0x55aa3f280ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.741+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 >> 172.21.15.47:0/3268431834 conn(0x55aa3f1a5e60 msgr2=0x55aa3f282590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:57.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.745+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 shutdown_connections 2024-05-11T12:26:57.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.746+0000 7f2797bf4b80 1 -- 172.21.15.47:0/3268431834 wait complete. 2024-05-11T12:26:57.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.747+0000 7f2797bf4b80 1 -- start start 2024-05-11T12:26:57.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.747+0000 7f2797bf4b80 1 Processor -- start 2024-05-11T12:26:57.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.748+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f299690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.749+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f299bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.749+0000 7f2797bf4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f27dd50 0x55aa3f30cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.749+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f277400c570 con 0x55aa3f279280 2024-05-11T12:26:57.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.749+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f277400d9e0 con 0x55aa3f27dd50 2024-05-11T12:26:57.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.749+0000 7f2797bf4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f277400cc40 con 0x55aa3f279ba0 2024-05-11T12:26:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.750+0000 7f2797bf4b80 10 client.0 _ll_register_callbacks cb 0x55aa3f163920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:26:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.751+0000 7f2786ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f299690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.751+0000 7f2786ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f299690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34086/0 (socket says 172.21.15.47:34086) 2024-05-11T12:26:57.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.751+0000 7f2786ffd700 1 -- 172.21.15.47:0/1482654522 learned_addr learned my addr 172.21.15.47:0/1482654522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:26:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.753+0000 7f2786ffd700 1 -- 172.21.15.47:0/1482654522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 msgr2=0x55aa3f299bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.753+0000 7f2786ffd700 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f299bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.753+0000 7f2786ffd700 1 -- 172.21.15.47:0/1482654522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f27dd50 msgr2=0x55aa3f30cd70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:26:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.753+0000 7f2786ffd700 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f27dd50 0x55aa3f30cd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.753+0000 7f2786ffd700 1 -- 172.21.15.47:0/1482654522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa3f30d2b0 con 0x55aa3f279280 2024-05-11T12:26:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.754+0000 7f2786ffd700 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f299690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2778006c20 tx=0x7f2778004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db44119436af83d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:26:57.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.754+0000 7f276aff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:26:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.754+0000 7f276aff5700 1 -- 172.21.15.47:0/1482654522 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778017040 con 0x55aa3f279280 2024-05-11T12:26:57.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.754+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f277c030420 con 0x55aa3f279280 2024-05-11T12:26:57.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.755+0000 7f276aff5700 1 -- 172.21.15.47:0/1482654522 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2778013880 con 0x55aa3f279280 2024-05-11T12:26:57.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.756+0000 7f276aff5700 1 -- 172.21.15.47:0/1482654522 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2778031050 con 0x55aa3f279280 2024-05-11T12:26:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.757+0000 7f2797bf4b80 10 client.15360 fetch_fsmap learned FSMap version 20 2024-05-11T12:26:57.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.757+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55aa3f30e4e0 con 0x55aa3f279280 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f276aff5700 1 -- 172.21.15.47:0/1482654522 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2778027070 con 0x55aa3f279280 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f276aff5700 1 -- 172.21.15.47:0/1482654522 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2778034020 con 0x55aa3f279280 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f2797bf4b80 10 client.15360 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f2797bf4b80 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f2797bf4b80 20 client.15360 populate_metadata read hostname 'smithi047' 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f30e930 0x55aa3f310d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f2797bf4b80 4 client.15360 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:26:57.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.761+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f277c030420 con 0x55aa3f30e930 2024-05-11T12:26:57.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.779+0000 7f27867fc700 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f30e930 0x55aa3f310d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:26:57.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.794+0000 7f27867fc700 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f30e930 0x55aa3f310d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a16474cd0ef1acc server_cookie=6647587292e3d9c1 in_seq=0 out_seq=0 2024-05-11T12:26:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.796+0000 7f276aff5700 10 client.15360 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.796+0000 7f276aff5700 1 -- 172.21.15.47:0/1482654522 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f277c030420 con 0x55aa3f30e930 2024-05-11T12:26:57.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.796+0000 7f276aff5700 10 client.15360 handle_command_reply: tid=0 2024-05-11T12:26:57.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.796+0000 7f2797bf4b80 1 client.15360 shutdown 2024-05-11T12:26:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 msgr2=0x55aa3f299690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f299690 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2778006c20 tx=0x7f2778004860 comp rx=0 tx=0).stop 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 shutdown_connections 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55aa3f30e930 0x55aa3f310d50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55aa3f27dd50 0x55aa3f30cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55aa3f279280 0x55aa3f299690 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 --2- 172.21.15.47:0/1482654522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55aa3f279ba0 0x55aa3f299bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 >> 172.21.15.47:0/1482654522 conn(0x55aa3f1a5e60 msgr2=0x55aa3f282590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f276aff5700 0 client.15360 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 shutdown_connections 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 1 -- 172.21.15.47:0/1482654522 wait complete. 2024-05-11T12:26:57.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.799+0000 7f2797bf4b80 20 client.15360 trim_cache size 0 max 16384 2024-05-11T12:26:57.802 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.802+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f27700b29c0 msgr2=0x7f27700b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.802+0000 7f2787fff700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f27700b29c0 0x7f27700b4e80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f277c006010 tx=0x7f277c0096f0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.802+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810dc00 msgr2=0x7f27881a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.802+0000 7f2787fff700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810dc00 0x7f27881a4ac0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f27800060b0 tx=0x7f278000d3b0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 shutdown_connections 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2788118550 0x7f27881a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f278810aa90 0x7f27881a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f27700b29c0 0x7f27700b4e80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 --2- 172.21.15.47:0/226900002 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f278810dc00 0x7f27881a4ac0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:26:57.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 >> 172.21.15.47:0/226900002 conn(0x7f278806e530 msgr2=0x7f2788118930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:26:57.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 shutdown_connections 2024-05-11T12:26:57.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:26:57.803+0000 7f2787fff700 1 -- 172.21.15.47:0/226900002 wait complete. 2024-05-11T12:26:59.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:26:58 smithi050 ceph-mon[34072]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 874 MiB/s wr, 1.65k op/s 2024-05-11T12:26:59.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:26:58 smithi047 ceph-mon[30375]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 874 MiB/s wr, 1.65k op/s 2024-05-11T12:26:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:26:58 smithi064 ceph-mon[34537]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 874 MiB/s wr, 1.65k op/s 2024-05-11T12:27:01.288 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:00 smithi047 ceph-mon[30375]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 795 MiB/s wr, 1.59k op/s 2024-05-11T12:27:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:00 smithi064 ceph-mon[34537]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 795 MiB/s wr, 1.59k op/s 2024-05-11T12:27:01.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:00 smithi050 ceph-mon[34072]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 795 MiB/s wr, 1.59k op/s 2024-05-11T12:27:01.574 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:27:01.574 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:27:02.608 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:27:03.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:02 smithi047 ceph-mon[30375]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 804 MiB/s wr, 1.78k op/s 2024-05-11T12:27:03.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:02 smithi064 ceph-mon[34537]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 804 MiB/s wr, 1.78k op/s 2024-05-11T12:27:03.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:02 smithi050 ceph-mon[34072]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 804 MiB/s wr, 1.78k op/s 2024-05-11T12:27:05.286 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:04 smithi047 ceph-mon[30375]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 2.40k op/s 2024-05-11T12:27:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:04 smithi064 ceph-mon[34537]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 2.40k op/s 2024-05-11T12:27:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:04 smithi050 ceph-mon[34072]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 MiB/s wr, 2.40k op/s 2024-05-11T12:27:07.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:06 smithi047 ceph-mon[30375]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 751 MiB/s wr, 2.46k op/s 2024-05-11T12:27:07.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:06 smithi064 ceph-mon[34537]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 751 MiB/s wr, 2.46k op/s 2024-05-11T12:27:07.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:06 smithi050 ceph-mon[34072]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 751 MiB/s wr, 2.46k op/s 2024-05-11T12:27:08.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.390+0000 7f52c5eb2700 1 -- 172.21.15.47:0/3400575116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c010a790 msgr2=0x7f52c010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.390+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/3400575116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c010a790 0x7f52c010e7e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52b40099c0 tx=0x7f52b4009cd0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.402+0000 7f52c5eb2700 1 -- 172.21.15.47:0/3400575116 shutdown_connections 2024-05-11T12:27:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.402+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/3400575116 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010eec0 0x7f52c0116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.402+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/3400575116 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c0109de0 0x7f52c010a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.402+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/3400575116 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c010a790 0x7f52c010e7e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.402+0000 7f52c5eb2700 1 -- 172.21.15.47:0/3400575116 >> 172.21.15.47:0/3400575116 conn(0x7f52c0076d20 msgr2=0x7f52c0079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:08.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.410+0000 7f52c5eb2700 1 -- 172.21.15.47:0/3400575116 shutdown_connections 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.411+0000 7f52c5eb2700 1 -- 172.21.15.47:0/3400575116 wait complete. 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.412+0000 7f52c5eb2700 1 Processor -- start 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.412+0000 7f52c5eb2700 1 -- start start 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.412+0000 7f52c5eb2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c0109de0 0x7f52c0074d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.412+0000 7f52c5eb2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010a790 0x7f52c006de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.413+0000 7f52c5eb2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c010eec0 0x7f52c006e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.413+0000 7f52c5eb2700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0119d90 con 0x7f52c010a790 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.413+0000 7f52c5eb2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0119c10 con 0x7f52c0109de0 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.413+0000 7f52c5eb2700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52c0119f10 con 0x7f52c010eec0 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.413+0000 7f52beffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010a790 0x7f52c006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.413+0000 7f52beffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010a790 0x7f52c006de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35654/0 (socket says 172.21.15.47:35654) 2024-05-11T12:27:08.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.413+0000 7f52beffd700 1 -- 172.21.15.47:0/2423971996 learned_addr learned my addr 172.21.15.47:0/2423971996 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:08.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.414+0000 7f52beffd700 1 -- 172.21.15.47:0/2423971996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c010eec0 msgr2=0x7f52c006e360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:27:08.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.414+0000 7f52beffd700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c010eec0 0x7f52c006e360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.414+0000 7f52beffd700 1 -- 172.21.15.47:0/2423971996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c0109de0 msgr2=0x7f52c0074d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.414+0000 7f52beffd700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c0109de0 0x7f52c0074d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.414+0000 7f52beffd700 1 -- 172.21.15.47:0/2423971996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c006e9d0 con 0x7f52c010a790 2024-05-11T12:27:08.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.415+0000 7f52beffd700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010a790 0x7f52c006de20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b580 tx=0x7f52b40037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.419+0000 7f52bcff9700 1 -- 172.21.15.47:0/2423971996 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4005da0 con 0x7f52c010a790 2024-05-11T12:27:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.419+0000 7f52bcff9700 1 -- 172.21.15.47:0/2423971996 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b400bd80 con 0x7f52c010a790 2024-05-11T12:27:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.419+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c006ec60 con 0x7f52c010a790 2024-05-11T12:27:08.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.419+0000 7f52bcff9700 1 -- 172.21.15.47:0/2423971996 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4005310 con 0x7f52c010a790 2024-05-11T12:27:08.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.419+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c0072ae0 con 0x7f52c010a790 2024-05-11T12:27:08.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.430+0000 7f52bcff9700 1 -- 172.21.15.47:0/2423971996 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f52b4004a50 con 0x7f52c010a790 2024-05-11T12:27:08.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.432+0000 7f52bcff9700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2a50 0x7f52a80b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.432+0000 7f52bcff9700 1 -- 172.21.15.47:0/2423971996 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f52b40e3110 con 0x7f52c010a790 2024-05-11T12:27:08.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.438+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2a50 0x7f52a80b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.442+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2a50 0x7f52a80b4f10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f52b80060b0 tx=0x7f52b8006040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:08.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.446+0000 7f52d0748b80 1 Processor -- start 2024-05-11T12:27:08.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.446+0000 7f52d0748b80 1 -- start start 2024-05-11T12:27:08.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.447+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda668f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.447+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda6635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.448+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda66f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.448+0000 7f52bf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda668f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.448+0000 7f52bf7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda668f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:60330/0 (socket says 172.21.15.47:60330) 2024-05-11T12:27:08.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.448+0000 7f52bf7fe700 1 -- 172.21.15.47:0/363583465 learned_addr learned my addr 172.21.15.47:0/363583465 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:08.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.448+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55beda5b58e0 con 0x55beda663ae0 2024-05-11T12:27:08.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.448+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55beda554840 con 0x55beda663120 2024-05-11T12:27:08.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.448+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55beda5b18a0 con 0x55beda668b50 2024-05-11T12:27:08.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.450+0000 7f52bffff700 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda66f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.452+0000 7f52bf7fe700 1 -- 172.21.15.47:0/363583465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 msgr2=0x55beda6635a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:27:08.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.452+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda6635a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.452+0000 7f52bf7fe700 1 -- 172.21.15.47:0/363583465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 msgr2=0x55beda66f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:08.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.452+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda66f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.452+0000 7f52bf7fe700 1 -- 172.21.15.47:0/363583465 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beda502950 con 0x55beda668b50 2024-05-11T12:27:08.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.453+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda668f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52b8007f70 tx=0x7f52b8010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef56872c7632a000 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:08.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.455+0000 7f52a5ffb700 1 -- 172.21.15.47:0/363583465 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8025070 con 0x55beda668b50 2024-05-11T12:27:08.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.455+0000 7f52a5ffb700 1 -- 172.21.15.47:0/363583465 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b8017da0 con 0x55beda668b50 2024-05-11T12:27:08.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.456+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 msgr2=0x55beda668f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.456+0000 7f52d0748b80 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda668f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52b8007f70 tx=0x7f52b8010000 comp rx=0 tx=0).stop 2024-05-11T12:27:08.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.461+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 shutdown_connections 2024-05-11T12:27:08.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.461+0000 7f52d0748b80 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda66f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.461+0000 7f52d0748b80 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda668f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.461+0000 7f52d0748b80 1 --2- 172.21.15.47:0/363583465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda6635a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.461+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 >> 172.21.15.47:0/363583465 conn(0x55beda58fee0 msgr2=0x55beda65da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:08.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.462+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 shutdown_connections 2024-05-11T12:27:08.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.467+0000 7f52d0748b80 1 -- 172.21.15.47:0/363583465 wait complete. 2024-05-11T12:27:08.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.468+0000 7f52d0748b80 1 -- start start 2024-05-11T12:27:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.468+0000 7f52d0748b80 1 Processor -- start 2024-05-11T12:27:08.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.469+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda66f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.469+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda66c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.469+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda66cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52d0748b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55beda66feb0 con 0x55beda663ae0 2024-05-11T12:27:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52bf7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda66f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52bf7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda66f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:38378/0 (socket says 172.21.15.47:38378) 2024-05-11T12:27:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52bf7fe700 1 -- 172.21.15.47:0/844481630 learned_addr learned my addr 172.21.15.47:0/844481630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52bffff700 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda66cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55beda51bcb0 con 0x55beda663120 2024-05-11T12:27:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55beda670030 con 0x55beda668b50 2024-05-11T12:27:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.470+0000 7f52d0748b80 10 client.0 _ll_register_callbacks cb 0x55beda54d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:08.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.478+0000 7f52bffff700 1 -- 172.21.15.47:0/844481630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 msgr2=0x55beda66f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:08.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.478+0000 7f52bffff700 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda66f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.478+0000 7f52bffff700 1 -- 172.21.15.47:0/844481630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 msgr2=0x55beda66c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:08.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.478+0000 7f52bffff700 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda66c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.478+0000 7f52bffff700 1 -- 172.21.15.47:0/844481630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beda551710 con 0x55beda668b50 2024-05-11T12:27:08.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.480+0000 7f52bffff700 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda66cf00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f52b000dca0 tx=0x7f52b000f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81e628ecd37b2aa6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.480+0000 7f529effd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:27:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.480+0000 7f529effd700 1 -- 172.21.15.47:0/844481630 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b000fea0 con 0x55beda668b50 2024-05-11T12:27:08.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.481+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55beda5c32c0 con 0x55beda668b50 2024-05-11T12:27:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.481+0000 7f529effd700 1 -- 172.21.15.47:0/844481630 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f52b0004830 con 0x55beda668b50 2024-05-11T12:27:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.481+0000 7f529effd700 1 -- 172.21.15.47:0/844481630 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f52b001f020 con 0x55beda668b50 2024-05-11T12:27:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.481+0000 7f52d0748b80 10 client.25021 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.481+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55beda67a730 con 0x55beda668b50 2024-05-11T12:27:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.482+0000 7f529effd700 1 -- 172.21.15.47:0/844481630 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55beda5c32c0 con 0x55beda668b50 2024-05-11T12:27:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.483+0000 7f529effd700 1 -- 172.21.15.47:0/844481630 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f52b0024650 con 0x55beda668b50 2024-05-11T12:27:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.484+0000 7f52d0748b80 10 client.25021 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.484+0000 7f52d0748b80 10 client.25021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.484+0000 7f52d0748b80 20 client.25021 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:08.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.484+0000 7f52d0748b80 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda67d0c0 0x55beda67f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.484+0000 7f52d0748b80 4 client.25021 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:27:08.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.484+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beda5bd360 con 0x55beda67d0c0 2024-05-11T12:27:08.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.487+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda67d0c0 0x55beda67f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.490+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda67d0c0 0x55beda67f4e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24df1c68d40bae5 server_cookie=ad3dda28ba8853b in_seq=0 out_seq=0 2024-05-11T12:27:08.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.490+0000 7f529effd700 10 client.25021 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:08.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.493+0000 7f529effd700 1 -- 172.21.15.47:0/844481630 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55beda5bd360 con 0x55beda67d0c0 2024-05-11T12:27:08.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.493+0000 7f529effd700 10 client.25021 handle_command_reply: tid=0 2024-05-11T12:27:08.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.493+0000 7f52d0748b80 1 client.25021 shutdown 2024-05-11T12:27:08.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.493+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 msgr2=0x55beda66cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.493+0000 7f52d0748b80 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda66cf00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f52b000dca0 tx=0x7f52b000f360 comp rx=0 tx=0).stop 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 shutdown_connections 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda67d0c0 0x55beda67f4e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda668b50 0x55beda66cf00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663120 0x55beda66f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 --2- 172.21.15.47:0/844481630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda66c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 >> 172.21.15.47:0/844481630 conn(0x55beda58fee0 msgr2=0x55beda6584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f529effd700 0 client.25021 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 shutdown_connections 2024-05-11T12:27:08.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.494+0000 7f52d0748b80 1 -- 172.21.15.47:0/844481630 wait complete. 2024-05-11T12:27:08.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.495+0000 7f52d0748b80 20 client.25021 trim_cache size 0 max 16384 2024-05-11T12:27:08.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.553+0000 7f52d0748b80 1 Processor -- start 2024-05-11T12:27:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.554+0000 7f52d0748b80 1 -- start start 2024-05-11T12:27:08.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.554+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 0x55beda678f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.554+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda662110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.554+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda668b50 0x55beda662650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.554+0000 7f52d0748b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01a9dc0 con 0x55beda663ae0 2024-05-11T12:27:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.554+0000 7f52d0748b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01ac590 con 0x55beda668b50 2024-05-11T12:27:08.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.554+0000 7f52d0748b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01aa380 con 0x55beda663120 2024-05-11T12:27:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.555+0000 7f52beffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda662110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.555+0000 7f52beffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda662110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35676/0 (socket says 172.21.15.47:35676) 2024-05-11T12:27:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.555+0000 7f52beffd700 1 -- 172.21.15.47:0/3891838015 learned_addr learned my addr 172.21.15.47:0/3891838015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.556+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 0x55beda678f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.556+0000 7f52bffff700 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda668b50 0x55beda662650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.557+0000 7f52bffff700 1 -- 172.21.15.47:0/3891838015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 msgr2=0x55beda678f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.557+0000 7f52bffff700 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 0x55beda678f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.557+0000 7f52bffff700 1 -- 172.21.15.47:0/3891838015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 msgr2=0x55beda662110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.557+0000 7f52bffff700 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda662110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.557+0000 7f52bffff700 1 -- 172.21.15.47:0/3891838015 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beda65a610 con 0x55beda668b50 2024-05-11T12:27:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.557+0000 7f52bffff700 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda668b50 0x55beda662650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f52b0024a20 tx=0x7f52b000dfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e087d7ea6adbb2d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.558+0000 7f52a57fa700 1 -- 172.21.15.47:0/3891838015 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0032070 con 0x55beda668b50 2024-05-11T12:27:08.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.558+0000 7f52a57fa700 1 -- 172.21.15.47:0/3891838015 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b0005570 con 0x55beda668b50 2024-05-11T12:27:08.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.558+0000 7f52a57fa700 1 -- 172.21.15.47:0/3891838015 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0003af0 con 0x55beda668b50 2024-05-11T12:27:08.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.560+0000 7f52d0748b80 1 -- 172.21.15.47:0/3891838015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda668b50 msgr2=0x55beda662650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.560+0000 7f52d0748b80 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda668b50 0x55beda662650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f52b0024a20 tx=0x7f52b000dfd0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.561+0000 7f52d0748b80 1 -- 172.21.15.47:0/3891838015 shutdown_connections 2024-05-11T12:27:08.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.561+0000 7f52d0748b80 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda668b50 0x55beda662650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.561+0000 7f52d0748b80 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 0x55beda678f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.561+0000 7f52d0748b80 1 --2- 172.21.15.47:0/3891838015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda663ae0 0x55beda662110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.561+0000 7f52d0748b80 1 -- 172.21.15.47:0/3891838015 >> 172.21.15.47:0/3891838015 conn(0x55beda58fee0 msgr2=0x55beda66b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.562+0000 7f52d0748b80 1 -- 172.21.15.47:0/3891838015 shutdown_connections 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.562+0000 7f52d0748b80 1 -- 172.21.15.47:0/3891838015 wait complete. 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.563+0000 7f52d0748b80 1 -- start start 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.563+0000 7f52d0748b80 1 Processor -- start 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.564+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 0x55beda6846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.564+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663ae0 0x55beda684c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.564+0000 7f52d0748b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda668b50 0x55beda685150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.564+0000 7f52d0748b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01aaa50 con 0x55beda668b50 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.564+0000 7f52d0748b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01abec0 con 0x55beda663ae0 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.564+0000 7f52d0748b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01ab120 con 0x55beda663120 2024-05-11T12:27:08.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.564+0000 7f52d0748b80 10 client.0 _ll_register_callbacks cb 0x55beda54d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:08.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.566+0000 7f52bffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda668b50 0x55beda685150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.566+0000 7f52bffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda668b50 0x55beda685150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35688/0 (socket says 172.21.15.47:35688) 2024-05-11T12:27:08.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.566+0000 7f52bffff700 1 -- 172.21.15.47:0/2502394842 learned_addr learned my addr 172.21.15.47:0/2502394842 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:08.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.567+0000 7f52bffff700 1 -- 172.21.15.47:0/2502394842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 msgr2=0x55beda6846d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:27:08.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.567+0000 7f52bffff700 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 0x55beda6846d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.567+0000 7f52bffff700 1 -- 172.21.15.47:0/2502394842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663ae0 msgr2=0x55beda684c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:27:08.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.567+0000 7f52bffff700 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663ae0 0x55beda684c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.567+0000 7f52bffff700 1 -- 172.21.15.47:0/2502394842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beda670070 con 0x55beda668b50 2024-05-11T12:27:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.572+0000 7f52bffff700 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda668b50 0x55beda685150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f52b000dc70 tx=0x7f52b0005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1deda717c333f54a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:08.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.572+0000 7f529effd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:27:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.573+0000 7f529effd700 1 -- 172.21.15.47:0/2502394842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0032070 con 0x55beda668b50 2024-05-11T12:27:08.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.574+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b0005570 con 0x55beda668b50 2024-05-11T12:27:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.574+0000 7f529effd700 1 -- 172.21.15.47:0/2502394842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f52b0003880 con 0x55beda668b50 2024-05-11T12:27:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.575+0000 7f529effd700 1 -- 172.21.15.47:0/2502394842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f52b0003a80 con 0x55beda668b50 2024-05-11T12:27:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.575+0000 7f52d0748b80 10 client.15375 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:08.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.575+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55beda6e7be0 con 0x55beda668b50 2024-05-11T12:27:08.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.576+0000 7f529effd700 1 -- 172.21.15.47:0/2502394842 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b0005570 con 0x55beda668b50 2024-05-11T12:27:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f529effd700 1 -- 172.21.15.47:0/2502394842 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f52b0040ab0 con 0x55beda668b50 2024-05-11T12:27:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f52d0748b80 10 client.15375 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f52d0748b80 10 client.15375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f52d0748b80 20 client.15375 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:08.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f52d0748b80 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda6e8130 0x55beda6ea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f52d0748b80 4 client.15375 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:27:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55beda4352d0 con 0x55beda6e8130 2024-05-11T12:27:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.577+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda6e8130 0x55beda6ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.578+0000 7f52bf7fe700 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda6e8130 0x55beda6ea560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf385f231f02dec server_cookie=de33d2e43fe957f7 in_seq=0 out_seq=0 2024-05-11T12:27:08.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.579+0000 7f529effd700 10 client.15375 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.583+0000 7f529effd700 1 -- 172.21.15.47:0/2502394842 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55beda4352d0 con 0x55beda6e8130 2024-05-11T12:27:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.583+0000 7f529effd700 10 client.15375 handle_command_reply: tid=0 2024-05-11T12:27:08.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.583+0000 7f52d0748b80 1 client.15375 shutdown 2024-05-11T12:27:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.583+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda668b50 msgr2=0x55beda685150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.583+0000 7f52d0748b80 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda668b50 0x55beda685150 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f52b000dc70 tx=0x7f52b0005500 comp rx=0 tx=0).stop 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 shutdown_connections 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55beda6e8130 0x55beda6ea560 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55beda668b50 0x55beda685150 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55beda663120 0x55beda6846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 --2- 172.21.15.47:0/2502394842 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55beda663ae0 0x55beda684c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 >> 172.21.15.47:0/2502394842 conn(0x55beda58fee0 msgr2=0x55beda662110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f529effd700 0 client.15375 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 shutdown_connections 2024-05-11T12:27:08.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.584+0000 7f52d0748b80 1 -- 172.21.15.47:0/2502394842 wait complete. 2024-05-11T12:27:08.586 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:27:08.587 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:27:08.587 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "31ad39db-62b7-4e60-a9d3-eb7961d8557d", 2024-05-11T12:27:08.587 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:27:08.587 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:27:08.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.586+0000 7f52d0748b80 20 client.15375 trim_cache size 0 max 16384 2024-05-11T12:27:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.589+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2a50 msgr2=0x7f52a80b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.589+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2a50 0x7f52a80b4f10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f52b80060b0 tx=0x7f52b8006040 comp rx=0 tx=0).stop 2024-05-11T12:27:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.589+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010a790 msgr2=0x7f52c006de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:08.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.589+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010a790 0x7f52c006de20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b580 tx=0x7f52b40037c0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 shutdown_connections 2024-05-11T12:27:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f52c010eec0 0x7f52c006e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f52a80b2a50 0x7f52a80b4f10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f52c0109de0 0x7f52c0074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 --2- 172.21.15.47:0/2423971996 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52c010a790 0x7f52c006de20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 >> 172.21.15.47:0/2423971996 conn(0x7f52c0076d20 msgr2=0x7f52c0078060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 shutdown_connections 2024-05-11T12:27:08.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:08.590+0000 7f52c5eb2700 1 -- 172.21.15.47:0/2423971996 wait complete. 2024-05-11T12:27:09.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:09 smithi047 ceph-mon[30375]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 825 MiB/s wr, 3.15k op/s 2024-05-11T12:27:09.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:09 smithi064 ceph-mon[34537]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 825 MiB/s wr, 3.15k op/s 2024-05-11T12:27:09.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:09 smithi050 ceph-mon[34072]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 825 MiB/s wr, 3.15k op/s 2024-05-11T12:27:10.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:27:10.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:10 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:27:10.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:10 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:10.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:10 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:27:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:10 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:27:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:10 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:10 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:27:10.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:10 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:27:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:10 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:10 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:27:11.179 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:27:11.326 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:11 smithi047 ceph-mon[30375]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 731 MiB/s wr, 3.21k op/s 2024-05-11T12:27:11.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:11 smithi064 ceph-mon[34537]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 731 MiB/s wr, 3.21k op/s 2024-05-11T12:27:11.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:11 smithi050 ceph-mon[34072]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 731 MiB/s wr, 3.21k op/s 2024-05-11T12:27:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:13 smithi064 ceph-mon[34537]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 759 MiB/s wr, 3.32k op/s 2024-05-11T12:27:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:13 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:13.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:13 smithi050 ceph-mon[34072]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 759 MiB/s wr, 3.32k op/s 2024-05-11T12:27:13.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:13 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:13.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:13 smithi047 ceph-mon[30375]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 759 MiB/s wr, 3.32k op/s 2024-05-11T12:27:13.515 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:13 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:14.346 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:14 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:27:14.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:14 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:27:14.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:14 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:27:15.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:15 smithi050 ceph-mon[34072]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 805 MiB/s wr, 3.18k op/s 2024-05-11T12:27:15.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:15 smithi047 ceph-mon[30375]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 805 MiB/s wr, 3.18k op/s 2024-05-11T12:27:15.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:15 smithi064 ceph-mon[34537]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 805 MiB/s wr, 3.18k op/s 2024-05-11T12:27:16.662 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:27:16.662 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:27:16.662 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:27:16.662 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.461+0000 7fcdcd355700 1 -- 172.21.15.47:0/1356853516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810a790 msgr2=0x7fcdc810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.461+0000 7fcdcd355700 1 --2- 172.21.15.47:0/1356853516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810a790 0x7fcdc810e7e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0099c0 tx=0x7fcdbc009cd0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.464+0000 7fcdcd355700 1 -- 172.21.15.47:0/1356853516 shutdown_connections 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.464+0000 7fcdcd355700 1 --2- 172.21.15.47:0/1356853516 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc810eec0 0x7fcdc8116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.464+0000 7fcdcd355700 1 --2- 172.21.15.47:0/1356853516 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdc8109de0 0x7fcdc810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.464+0000 7fcdcd355700 1 --2- 172.21.15.47:0/1356853516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810a790 0x7fcdc810e7e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.464+0000 7fcdcd355700 1 -- 172.21.15.47:0/1356853516 >> 172.21.15.47:0/1356853516 conn(0x7fcdc8076d20 msgr2=0x7fcdc8108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.469+0000 7fcdcd355700 1 -- 172.21.15.47:0/1356853516 shutdown_connections 2024-05-11T12:27:16.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.469+0000 7fcdcd355700 1 -- 172.21.15.47:0/1356853516 wait complete. 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.470+0000 7fcdcd355700 1 Processor -- start 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.470+0000 7fcdcd355700 1 -- start start 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.471+0000 7fcdcd355700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdc8109de0 0x7fcdc8074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.471+0000 7fcdcd355700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc810a790 0x7fcdc806de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.471+0000 7fcdcd355700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810eec0 0x7fcdc806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.471+0000 7fcdcd355700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8119e30 con 0x7fcdc810eec0 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.471+0000 7fcdcd355700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8119cb0 con 0x7fcdc8109de0 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.471+0000 7fcdcd355700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8119fb0 con 0x7fcdc810a790 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.472+0000 7fcdc77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810eec0 0x7fcdc806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.472+0000 7fcdc77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810eec0 0x7fcdc806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35698/0 (socket says 172.21.15.47:35698) 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.472+0000 7fcdc67fc700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc810a790 0x7fcdc806de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.472+0000 7fcdc77fe700 1 -- 172.21.15.47:0/3212838623 learned_addr learned my addr 172.21.15.47:0/3212838623 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.473+0000 7fcdc77fe700 1 -- 172.21.15.47:0/3212838623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc810a790 msgr2=0x7fcdc806de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.473+0000 7fcdc77fe700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc810a790 0x7fcdc806de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.473+0000 7fcdc77fe700 1 -- 172.21.15.47:0/3212838623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdc8109de0 msgr2=0x7fcdc8074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.473+0000 7fcdc77fe700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdc8109de0 0x7fcdc8074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.473+0000 7fcdc77fe700 1 -- 172.21.15.47:0/3212838623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc806e9f0 con 0x7fcdc810eec0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.473+0000 7fcdc77fe700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810eec0 0x7fcdc806e3b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c970 tx=0x7fcdb800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.474+0000 7fcdaffff700 1 -- 172.21.15.47:0/3212838623 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8007b00 con 0x7fcdc810eec0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.474+0000 7fcdaffff700 1 -- 172.21.15.47:0/3212838623 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb8004ce0 con 0x7fcdc810eec0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.474+0000 7fcdaffff700 1 -- 172.21.15.47:0/3212838623 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8005530 con 0x7fcdc810eec0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.475+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc8072870 con 0x7fcdc810eec0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.475+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc8072d30 con 0x7fcdc810eec0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.486+0000 7fcdaffff700 1 -- 172.21.15.47:0/3212838623 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fcdb8004750 con 0x7fcdc810eec0 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.488+0000 7fcdd7bebb80 1 Processor -- start 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.488+0000 7fcdd7bebb80 1 -- start start 2024-05-11T12:27:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.488+0000 7fcdaffff700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcdb00b2980 0x7fcdb00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.489+0000 7fcdaffff700 1 -- 172.21.15.47:0/3212838623 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fcdb80e1f50 con 0x7fcdc810eec0 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.489+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8459eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.489+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d84540a0 0x55c0d8454520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.489+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d8460890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.489+0000 7fcdd7bebb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c0d83a6860 con 0x55c0d8459ad0 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.490+0000 7fcdd7bebb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c0d83457c0 con 0x55c0d84540a0 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.490+0000 7fcdd7bebb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0d83a2820 con 0x55c0d8454a60 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.490+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcdb00b2980 0x7fcdb00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.491+0000 7fcdc77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d8460890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.491+0000 7fcdc77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d8460890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:60380/0 (socket says 172.21.15.47:60380) 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.491+0000 7fcdc77fe700 1 -- 172.21.15.47:0/2581572444 learned_addr learned my addr 172.21.15.47:0/2581572444 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:16.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.491+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8459eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.491+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d84540a0 0x55c0d8454520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.491+0000 7fcdc77fe700 1 -- 172.21.15.47:0/2581572444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d84540a0 msgr2=0x55c0d8454520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdc77fe700 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d84540a0 0x55c0d8454520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdc77fe700 1 -- 172.21.15.47:0/2581572444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 msgr2=0x55c0d8459eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdc77fe700 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8459eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdc77fe700 1 -- 172.21.15.47:0/2581572444 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d82f38d0 con 0x55c0d8454a60 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d84540a0 0x55c0d8454520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdc77fe700 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d8460890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80a54c0 tx=0x7fcdb8005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5a28375ea5cf5e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdad7fa700 1 -- 172.21.15.47:0/2581572444 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb801d040 con 0x55c0d8454a60 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.492+0000 7fcdad7fa700 1 -- 172.21.15.47:0/2581572444 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdb8007ca0 con 0x55c0d8454a60 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.493+0000 7fcdad7fa700 1 -- 172.21.15.47:0/2581572444 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80ad7f0 con 0x55c0d8454a60 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2581572444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 msgr2=0x55c0d8460890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d8460890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80a54c0 tx=0x7fcdb8005870 comp rx=0 tx=0).stop 2024-05-11T12:27:16.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcdb00b2980 0x7fcdb00b4e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009990 tx=0x7fcdbc00b7d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2581572444 shutdown_connections 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d8460890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8459eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2581572444 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d84540a0 0x55c0d8454520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2581572444 >> 172.21.15.47:0/2581572444 conn(0x55c0d8380e60 msgr2=0x55c0d844e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.494+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2581572444 shutdown_connections 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.495+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2581572444 wait complete. 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.496+0000 7fcdd7bebb80 1 -- start start 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.496+0000 7fcdd7bebb80 1 Processor -- start 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.498+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d84540a0 0x55c0d84604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.498+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d845d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.498+0000 7fcdc6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d84540a0 0x55c0d84604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8459ad0 0x55c0d845dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdd7bebb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55c0d8460e90 con 0x55c0d84540a0 2024-05-11T12:27:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdd7bebb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55c0d830cc30 con 0x55c0d8459ad0 2024-05-11T12:27:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdd7bebb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55c0d8461010 con 0x55c0d8454a60 2024-05-11T12:27:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdd7bebb80 10 client.0 _ll_register_callbacks cb 0x55c0d833e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdc6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d84540a0 0x55c0d84604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35718/0 (socket says 172.21.15.47:35718) 2024-05-11T12:27:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/2735028303 learned_addr learned my addr 172.21.15.47:0/2735028303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.499+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d845d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.500+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/2735028303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 msgr2=0x55c0d845d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.500+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d845d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.500+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/2735028303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8459ad0 msgr2=0x55c0d845dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.500+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8459ad0 0x55c0d845dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.500+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/2735028303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d8342690 con 0x55c0d84540a0 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.500+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d84540a0 0x55c0d84604c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000aad0 tx=0x7fcdc000ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700f79e74fdc6e9c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.500+0000 7fcdaaff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.501+0000 7fcdaaff5700 1 -- 172.21.15.47:0/2735028303 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0018020 con 0x55c0d84540a0 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.501+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c0d83b4240 con 0x55c0d84540a0 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.501+0000 7fcdaaff5700 1 -- 172.21.15.47:0/2735028303 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fcdc000ea80 con 0x55c0d84540a0 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.501+0000 7fcdaaff5700 1 -- 172.21.15.47:0/2735028303 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcdc0023050 con 0x55c0d84540a0 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.501+0000 7fcdd7bebb80 10 client.15387 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:16.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.501+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c0d846b6d0 con 0x55c0d84540a0 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdaaff5700 1 -- 172.21.15.47:0/2735028303 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c0d83b4240 con 0x55c0d84540a0 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdaaff5700 1 -- 172.21.15.47:0/2735028303 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fcdc001f490 con 0x55c0d84540a0 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdd7bebb80 10 client.15387 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdd7bebb80 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdd7bebb80 20 client.15387 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d846e060 0x55c0d8470500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdd7bebb80 4 client.15387 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.502+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0d83ae2e0 con 0x55c0d846e060 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.503+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d846e060 0x55c0d8470500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.506+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d846e060 0x55c0d8470500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bacba4d2b4386aca server_cookie=e872e23c7afe1b8f in_seq=0 out_seq=0 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.506+0000 7fcdaaff5700 10 client.15387 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdaaff5700 1 -- 172.21.15.47:0/2735028303 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c0d83ae2e0 con 0x55c0d846e060 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdaaff5700 10 client.15387 handle_command_reply: tid=0 2024-05-11T12:27:16.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 client.15387 shutdown 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d84540a0 msgr2=0x55c0d84604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d84540a0 0x55c0d84604c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000aad0 tx=0x7fcdc000ae90 comp rx=0 tx=0).stop 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 shutdown_connections 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d846e060 0x55c0d8470500 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8459ad0 0x55c0d845dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d84540a0 0x55c0d84604c0 secure :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000aad0 tx=0x7fcdc000ae90 comp rx=0 tx=0).stop 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/2735028303 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8454a60 0x55c0d845d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 >> 172.21.15.47:0/2735028303 conn(0x55c0d8380e60 msgr2=0x55c0d8449460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdaaff5700 0 client.15387 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.508+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 shutdown_connections 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.509+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/2735028303 wait complete. 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.509+0000 7fcdd7bebb80 20 client.15387 trim_cache size 0 max 16384 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.539+0000 7fcdd7bebb80 1 Processor -- start 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.539+0000 7fcdd7bebb80 1 -- start start 2024-05-11T12:27:16.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.539+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8454a60 0x55c0d845c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.540+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8469ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.540+0000 7fcdc6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8454a60 0x55c0d845c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.540+0000 7fcdc6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8454a60 0x55c0d845c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:38432/0 (socket says 172.21.15.47:38432) 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.540+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/72125606 learned_addr learned my addr 172.21.15.47:0/72125606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.540+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8457c60 0x55c0d845fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.541+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8469ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.541+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8080970 con 0x55c0d8459ad0 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.541+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc80831f0 con 0x55c0d8454a60 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.541+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8080fe0 con 0x55c0d8457c60 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.542+0000 7fcdc77fe700 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8457c60 0x55c0d845fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.543+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/72125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8457c60 msgr2=0x55c0d845fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.543+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8457c60 0x55c0d845fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.543+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/72125606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 msgr2=0x55c0d8469ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.543+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8469ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.543+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/72125606 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d8450240 con 0x55c0d8454a60 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.545+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8454a60 0x55c0d845c820 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcdc001f8d0 tx=0x7fcdc000e470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ede5b7f354eab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.545+0000 7fcdacff9700 1 -- 172.21.15.47:0/72125606 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc002b020 con 0x55c0d8454a60 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.545+0000 7fcdacff9700 1 -- 172.21.15.47:0/72125606 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdc001fcc0 con 0x55c0d8454a60 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.546+0000 7fcdacff9700 1 -- 172.21.15.47:0/72125606 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0018070 con 0x55c0d8454a60 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.547+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8454a60 msgr2=0x55c0d845c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.547+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8454a60 0x55c0d845c820 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcdc001f8d0 tx=0x7fcdc000e470 comp rx=0 tx=0).stop 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.547+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 shutdown_connections 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.547+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8457c60 0x55c0d845fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.547+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8454a60 0x55c0d845c820 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.547+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/72125606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8459ad0 0x55c0d8469ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.548+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 >> 172.21.15.47:0/72125606 conn(0x55c0d8380e60 msgr2=0x55c0d8457550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.548+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 shutdown_connections 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.548+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/72125606 wait complete. 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.549+0000 7fcdd7bebb80 1 -- start start 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.549+0000 7fcdd7bebb80 1 Processor -- start 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8454a60 0x55c0d8466440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdd7bebb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8457c60 0x55c0d8466980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdc6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8454a60 0x55c0d8466440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdc6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8454a60 0x55c0d8466440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35740/0 (socket says 172.21.15.47:35740) 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/390128020 learned_addr learned my addr 172.21.15.47:0/390128020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8457c60 0x55c0d8466980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8459ad0 0x55c0d8463430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc80816b0 con 0x55c0d8454a60 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.550+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8082b20 con 0x55c0d8457c60 2024-05-11T12:27:16.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.551+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8081d80 con 0x55c0d8459ad0 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.551+0000 7fcdd7bebb80 10 client.0 _ll_register_callbacks cb 0x55c0d833e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.551+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/390128020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8459ad0 msgr2=0x55c0d8463430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.551+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8459ad0 0x55c0d8463430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.551+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8457c60 msgr2=0x55c0d8466980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.551+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8457c60 0x55c0d8466980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.551+0000 7fcdc6ffd700 1 -- 172.21.15.47:0/390128020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d8463c40 con 0x55c0d8454a60 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.552+0000 7fcdc6ffd700 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8454a60 0x55c0d8466440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001220 tx=0x7fcdc000a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14311578f469367 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.552+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdc001fcc0 con 0x55c0d8454a60 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.552+0000 7fcdaaff5700 10 client.15393 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.552+0000 7fcdaaff5700 1 -- 172.21.15.47:0/390128020 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc002b030 con 0x55c0d8454a60 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.553+0000 7fcdaaff5700 1 -- 172.21.15.47:0/390128020 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fcdc000ee30 con 0x55c0d8454a60 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.553+0000 7fcdaaff5700 1 -- 172.21.15.47:0/390128020 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcdc0018050 con 0x55c0d8454a60 2024-05-11T12:27:16.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.554+0000 7fcdd7bebb80 10 client.15393 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.554+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55c0d84d9560 con 0x55c0d8454a60 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.554+0000 7fcdaaff5700 1 -- 172.21.15.47:0/390128020 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fcdc0003aa0 con 0x55c0d8454a60 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.554+0000 7fcdd7bebb80 10 client.15393 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.554+0000 7fcdd7bebb80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.555+0000 7fcdd7bebb80 20 client.15393 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.555+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d84d9ab0 0x55c0d84dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.555+0000 7fcdd7bebb80 4 client.15393 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.555+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fcdc001fcc0 con 0x55c0d84d9ab0 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.555+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d84d9ab0 0x55c0d84dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.556+0000 7fcdc67fc700 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d84d9ab0 0x55c0d84dbef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be6766f9f3b01a7 server_cookie=bc8d64ed5b8783c0 in_seq=0 out_seq=0 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.556+0000 7fcdaaff5700 10 client.15393 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.557+0000 7fcdaaff5700 1 -- 172.21.15.47:0/390128020 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fcdc001fcc0 con 0x55c0d84d9ab0 2024-05-11T12:27:16.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.557+0000 7fcdaaff5700 10 client.15393 handle_command_reply: tid=0 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.557+0000 7fcdd7bebb80 1 client.15393 shutdown 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.564+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8454a60 msgr2=0x55c0d8466440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.564+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8454a60 0x55c0d8466440 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001220 tx=0x7fcdc000a9c0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 shutdown_connections 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55c0d84d9ab0 0x55c0d84dbef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55c0d8459ad0 0x55c0d8463430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55c0d8454a60 0x55c0d8466440 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdaaff5700 0 client.15393 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 --2- 172.21.15.47:0/390128020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55c0d8457c60 0x55c0d8466980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 >> 172.21.15.47:0/390128020 conn(0x55c0d8380e60 msgr2=0x55c0d845a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 shutdown_connections 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 1 -- 172.21.15.47:0/390128020 wait complete. 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.565+0000 7fcdd7bebb80 20 client.15393 trim_cache size 0 max 16384 2024-05-11T12:27:16.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.567+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcdb00b2980 msgr2=0x7fcdb00b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.567+0000 7fcdcd355700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcdb00b2980 0x7fcdb00b4e40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc009990 tx=0x7fcdbc00b7d0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.567+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810eec0 msgr2=0x7fcdc806e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.567+0000 7fcdcd355700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810eec0 0x7fcdc806e3b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800c970 tx=0x7fcdb800cd30 comp rx=0 tx=0).stop 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.574+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 shutdown_connections 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.574+0000 7fcdcd355700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdc810eec0 0x7fcdc806e3b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.574+0000 7fcdcd355700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fcdb00b2980 0x7fcdb00b4e40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.574+0000 7fcdcd355700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcdc8109de0 0x7fcdc8074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.574+0000 7fcdcd355700 1 --2- 172.21.15.47:0/3212838623 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fcdc810a790 0x7fcdc806de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.574+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 >> 172.21.15.47:0/3212838623 conn(0x7fcdc8076d20 msgr2=0x7fcdc8110090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.575+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 shutdown_connections 2024-05-11T12:27:16.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:16.575+0000 7fcdcd355700 1 -- 172.21.15.47:0/3212838623 wait complete. 2024-05-11T12:27:17.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:17 smithi050 ceph-mon[34072]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 673 MiB/s wr, 2.40k op/s 2024-05-11T12:27:17.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:17 smithi047 ceph-mon[30375]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 673 MiB/s wr, 2.40k op/s 2024-05-11T12:27:17.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:17 smithi064 ceph-mon[34537]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 673 MiB/s wr, 2.40k op/s 2024-05-11T12:27:18.996 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:27:18.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:27:19.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:19 smithi047 ceph-mon[30375]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 821 MiB/s wr, 2.39k op/s 2024-05-11T12:27:19.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:19 smithi064 ceph-mon[34537]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 821 MiB/s wr, 2.39k op/s 2024-05-11T12:27:19.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:19 smithi050 ceph-mon[34072]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 821 MiB/s wr, 2.39k op/s 2024-05-11T12:27:20.125 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:27:21.504 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:21 smithi047 ceph-mon[30375]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 755 MiB/s wr, 1.74k op/s 2024-05-11T12:27:21.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:21 smithi064 ceph-mon[34537]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 755 MiB/s wr, 1.74k op/s 2024-05-11T12:27:21.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:21 smithi050 ceph-mon[34072]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 755 MiB/s wr, 1.74k op/s 2024-05-11T12:27:22.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:22 smithi047 ceph-mon[30375]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 809 MiB/s wr, 1.76k op/s 2024-05-11T12:27:22.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:22 smithi064 ceph-mon[34537]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 809 MiB/s wr, 1.76k op/s 2024-05-11T12:27:22.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:22 smithi050 ceph-mon[34072]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 809 MiB/s wr, 1.76k op/s 2024-05-11T12:27:24.947 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:24 smithi047 ceph-mon[30375]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 854 MiB/s wr, 1.90k op/s 2024-05-11T12:27:25.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:24 smithi064 ceph-mon[34537]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 854 MiB/s wr, 1.90k op/s 2024-05-11T12:27:25.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:24 smithi050 ceph-mon[34072]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 854 MiB/s wr, 1.90k op/s 2024-05-11T12:27:25.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.240+0000 7fc17c0ee700 1 -- 172.21.15.47:0/3890079713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410eec0 msgr2=0x7fc174116a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:25.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.241+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/3890079713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410eec0 0x7fc174116a90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00b0d0 tx=0x7fc16c00b3e0 comp rx=0 tx=0).stop 2024-05-11T12:27:25.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.241+0000 7fc17c0ee700 1 -- 172.21.15.47:0/3890079713 shutdown_connections 2024-05-11T12:27:25.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.241+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/3890079713 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410eec0 0x7fc174116a90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.241+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/3890079713 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc174109de0 0x7fc17410a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.241+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/3890079713 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc17410a790 0x7fc17410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.242+0000 7fc17c0ee700 1 -- 172.21.15.47:0/3890079713 >> 172.21.15.47:0/3890079713 conn(0x7fc174076d20 msgr2=0x7fc174079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.242+0000 7fc17c0ee700 1 -- 172.21.15.47:0/3890079713 shutdown_connections 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.243+0000 7fc17c0ee700 1 -- 172.21.15.47:0/3890079713 wait complete. 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.244+0000 7fc17c0ee700 1 Processor -- start 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.245+0000 7fc17c0ee700 1 -- start start 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.246+0000 7fc17c0ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc174109de0 0x7fc1741a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.246+0000 7fc17c0ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410a790 0x7fc1741a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.246+0000 7fc17c0ee700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc17410eec0 0x7fc1741a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.246+0000 7fc17c0ee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc174119e50 con 0x7fc17410a790 2024-05-11T12:27:25.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.246+0000 7fc17c0ee700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc174119cd0 con 0x7fc17410eec0 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc17c0ee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc174119fd0 con 0x7fc174109de0 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410a790 0x7fc1741a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410a790 0x7fc1741a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37426/0 (socket says 172.21.15.47:37426) 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 -- 172.21.15.47:0/1275753127 learned_addr learned my addr 172.21.15.47:0/1275753127 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc17a68b700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc17410eec0 0x7fc1741a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 -- 172.21.15.47:0/1275753127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc174109de0 msgr2=0x7fc1741a0420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc174109de0 0x7fc1741a0420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 -- 172.21.15.47:0/1275753127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc17410eec0 msgr2=0x7fc1741a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc17410eec0 0x7fc1741a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.247+0000 7fc179689700 1 -- 172.21.15.47:0/1275753127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1741a53e0 con 0x7fc17410a790 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.248+0000 7fc179689700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410a790 0x7fc1741a0960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc16400b500 tx=0x7fc16400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.249+0000 7fc16affd700 1 -- 172.21.15.47:0/1275753127 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164015020 con 0x7fc17410a790 2024-05-11T12:27:25.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.249+0000 7fc16affd700 1 -- 172.21.15.47:0/1275753127 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc164011ce0 con 0x7fc17410a790 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.249+0000 7fc16affd700 1 -- 172.21.15.47:0/1275753127 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164010d40 con 0x7fc17410a790 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.249+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1741a56d0 con 0x7fc17410a790 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.249+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1741acf50 con 0x7fc17410a790 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.251+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17406a240 con 0x7fc17410a790 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.254+0000 7fc16affd700 1 -- 172.21.15.47:0/1275753127 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc164018020 con 0x7fc17410a790 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.256+0000 7fc16affd700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1600b29d0 0x7fc1600b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.256+0000 7fc16affd700 1 -- 172.21.15.47:0/1275753127 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc1640e1ea0 con 0x7fc17410a790 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.256+0000 7fc179e8a700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1600b29d0 0x7fc1600b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.257+0000 7fc179e8a700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1600b29d0 0x7fc1600b4e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc170009730 tx=0x7fc170006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:25.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:25.264+0000 7fc16affd700 1 -- 172.21.15.47:0/1275753127 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc16401f020 con 0x7fc17410a790 2024-05-11T12:27:26.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.006+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc17402e570 con 0x7fc17410a790 2024-05-11T12:27:26.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.010+0000 7fc16affd700 1 -- 172.21.15.47:0/1275753127 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fc1640a9b70 con 0x7fc17410a790 2024-05-11T12:27:26.012 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:27:26.012 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:27:26.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.020+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1600b29d0 msgr2=0x7fc1600b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:26.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.020+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1600b29d0 0x7fc1600b4e90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc170009730 tx=0x7fc170006cb0 comp rx=0 tx=0).stop 2024-05-11T12:27:26.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.020+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410a790 msgr2=0x7fc1741a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:26.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.020+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410a790 0x7fc1741a0960 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc16400b500 tx=0x7fc16400b810 comp rx=0 tx=0).stop 2024-05-11T12:27:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.022+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 shutdown_connections 2024-05-11T12:27:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.022+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1600b29d0 0x7fc1600b4e90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.022+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc17410eec0 0x7fc1741a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:26.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.022+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc174109de0 0x7fc1741a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.022+0000 7fc17c0ee700 1 --2- 172.21.15.47:0/1275753127 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc17410a790 0x7fc1741a0960 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:26.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.022+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 >> 172.21.15.47:0/1275753127 conn(0x7fc174076d20 msgr2=0x7fc1740790d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.022+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 shutdown_connections 2024-05-11T12:27:26.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:26.024+0000 7fc17c0ee700 1 -- 172.21.15.47:0/1275753127 wait complete. 2024-05-11T12:27:26.042 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:27:27.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:26 smithi064 ceph-mon[34537]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 735 MiB/s wr, 1.74k op/s 2024-05-11T12:27:27.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:26 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1275753127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:27:27.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:26 smithi047 ceph-mon[30375]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 735 MiB/s wr, 1.74k op/s 2024-05-11T12:27:27.195 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:26 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1275753127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:27:27.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:26 smithi050 ceph-mon[34072]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 735 MiB/s wr, 1.74k op/s 2024-05-11T12:27:27.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:26 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1275753127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:27:29.033 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:27:29.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:28 smithi064 ceph-mon[34537]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 912 MiB/s wr, 1.97k op/s 2024-05-11T12:27:29.163 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:28 smithi047 ceph-mon[30375]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 912 MiB/s wr, 1.97k op/s 2024-05-11T12:27:29.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:28 smithi050 ceph-mon[34072]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 912 MiB/s wr, 1.97k op/s 2024-05-11T12:27:30.273 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:27:31.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:30 smithi064 ceph-mon[34537]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.76k op/s 2024-05-11T12:27:31.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:30 smithi050 ceph-mon[34072]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.76k op/s 2024-05-11T12:27:31.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:30 smithi047 ceph-mon[30375]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 769 MiB/s wr, 1.76k op/s 2024-05-11T12:27:33.113 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:32 smithi064 ceph-mon[34537]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 838 MiB/s wr, 2.06k op/s 2024-05-11T12:27:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:32 smithi050 ceph-mon[34072]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 838 MiB/s wr, 2.06k op/s 2024-05-11T12:27:33.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:32 smithi047 ceph-mon[30375]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 838 MiB/s wr, 2.06k op/s 2024-05-11T12:27:34.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:33 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:27:34.240 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:33 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:27:34.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:33 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:27:35.168 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:34 smithi047 ceph-mon[30375]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 859 MiB/s wr, 2.09k op/s 2024-05-11T12:27:35.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:34 smithi050 ceph-mon[34072]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 859 MiB/s wr, 2.09k op/s 2024-05-11T12:27:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:34 smithi064 ceph-mon[34537]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 859 MiB/s wr, 2.09k op/s 2024-05-11T12:27:35.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.690+0000 7fb7f5882700 1 -- 172.21.15.47:0/494881198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80b7af0 msgr2=0x7fb7e80b9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.690+0000 7fb7f5882700 1 --2- 172.21.15.47:0/494881198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80b9ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0098f0 tx=0x7fb7dc009c00 comp rx=0 tx=0).stop 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.692+0000 7fb7f5882700 1 -- 172.21.15.47:0/494881198 shutdown_connections 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.692+0000 7fb7f5882700 1 --2- 172.21.15.47:0/494881198 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80b9ee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.692+0000 7fb7f5882700 1 --2- 172.21.15.47:0/494881198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e80a4dd0 0x7fb7e80a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.692+0000 7fb7f5882700 1 --2- 172.21.15.47:0/494881198 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80a56f0 0x7fb7e80b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.692+0000 7fb7f5882700 1 -- 172.21.15.47:0/494881198 >> 172.21.15.47:0/494881198 conn(0x7fb7e801a690 msgr2=0x7fb7e801aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.693+0000 7fb7f5882700 1 -- 172.21.15.47:0/494881198 shutdown_connections 2024-05-11T12:27:35.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.693+0000 7fb7f5882700 1 -- 172.21.15.47:0/494881198 wait complete. 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.694+0000 7fb7f5882700 1 Processor -- start 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.695+0000 7fb7f5882700 1 -- start start 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.695+0000 7fb7f5882700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80a4dd0 0x7fb7e80b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.695+0000 7fb7f5882700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e80a56f0 0x7fb7e80aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.695+0000 7fb7f5882700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.696+0000 7fb7f5882700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e80bd510 con 0x7fb7e80a56f0 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.696+0000 7fb7f5882700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e80bd390 con 0x7fb7e80b7af0 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.696+0000 7fb7f5882700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e80bd690 con 0x7fb7e80a4dd0 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.697+0000 7fb7f5081700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.697+0000 7fb7f5081700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:60918/0 (socket says 172.21.15.47:60918) 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.697+0000 7fb7f5081700 1 -- 172.21.15.47:0/1136925305 learned_addr learned my addr 172.21.15.47:0/1136925305 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:35.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.697+0000 7fb7f4880700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80a4dd0 0x7fb7e80b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.703+0000 7fb7effff700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e80a56f0 0x7fb7e80aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.703+0000 7fb7f5081700 1 -- 172.21.15.47:0/1136925305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80a4dd0 msgr2=0x7fb7e80b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.703+0000 7fb7f5081700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80a4dd0 0x7fb7e80b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.703+0000 7fb7f5081700 1 -- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e80a56f0 msgr2=0x7fb7e80aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.703+0000 7fb7f5081700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e80a56f0 0x7fb7e80aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.703+0000 7fb7f5081700 1 -- 172.21.15.47:0/1136925305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e80af780 con 0x7fb7e80b7af0 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.708+0000 7fb7f5081700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80af140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00b850 tx=0x7fb7dc00b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.710+0000 7fb7effff700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e80a56f0 0x7fb7e80aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.710+0000 7fb7edffb700 1 -- 172.21.15.47:0/1136925305 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc025070 con 0x7fb7e80b7af0 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.711+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e80afa10 con 0x7fb7e80b7af0 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.711+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e80b3980 con 0x7fb7e80b7af0 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.718+0000 7fb7edffb700 1 -- 172.21.15.47:0/1136925305 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7dc00bd90 con 0x7fb7e80b7af0 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.718+0000 7fb7edffb700 1 -- 172.21.15.47:0/1136925305 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc004df0 con 0x7fb7e80b7af0 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.725+0000 7fb80137ab80 1 Processor -- start 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.730+0000 7fb80137ab80 1 -- start start 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.733+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdeeb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.733+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdeef810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.733+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.733+0000 7fb80137ab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e0cde3d860 con 0x55e0cdeebba0 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.733+0000 7fb80137ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdddc7c0 con 0x55e0cdeeb280 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.733+0000 7fb80137ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0cde39820 con 0x55e0cdeefd50 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.733+0000 7fb7edffb700 1 -- 172.21.15.47:0/1136925305 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb7dc028070 con 0x7fb7e80b7af0 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.734+0000 7fb7f4880700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdeef810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.734+0000 7fb7f4880700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdeef810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38342/0 (socket says 172.21.15.47:38342) 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.734+0000 7fb7f4880700 1 -- 172.21.15.47:0/1482913000 learned_addr learned my addr 172.21.15.47:0/1482913000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.735+0000 7fb7effff700 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.735+0000 7fb7edffb700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb7e00b2a50 0x7fb7e00b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.735+0000 7fb7effff700 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdeeb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.735+0000 7fb7edffb700 1 -- 172.21.15.47:0/1136925305 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb7dc0e2c10 con 0x7fb7e80b7af0 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.737+0000 7fb7f4880700 1 -- 172.21.15.47:0/1482913000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 msgr2=0x55e0cdef0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.737+0000 7fb7f4880700 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.737+0000 7fb7f4880700 1 -- 172.21.15.47:0/1482913000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 msgr2=0x55e0cdeeb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.737+0000 7fb7f4880700 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdeeb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.738+0000 7fb7f4880700 1 -- 172.21.15.47:0/1482913000 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0cdd8a8d0 con 0x55e0cdeebba0 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.742+0000 7fb7f4880700 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdeef810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400bd30 tx=0x7fb7e400f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff005062df43a34 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.743+0000 7fb7f5081700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb7e00b2a50 0x7fb7e00b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.744+0000 7fb7daffd700 1 -- 172.21.15.47:0/1482913000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4015070 con 0x55e0cdeebba0 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.744+0000 7fb7daffd700 1 -- 172.21.15.47:0/1482913000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e400fe50 con 0x55e0cdeebba0 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.744+0000 7fb7daffd700 1 -- 172.21.15.47:0/1482913000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4011710 con 0x55e0cdeebba0 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.746+0000 7fb80137ab80 1 -- 172.21.15.47:0/1482913000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 msgr2=0x55e0cdeef810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.746+0000 7fb80137ab80 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdeef810 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400bd30 tx=0x7fb7e400f640 comp rx=0 tx=0).stop 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.747+0000 7fb80137ab80 1 -- 172.21.15.47:0/1482913000 shutdown_connections 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.747+0000 7fb80137ab80 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.747+0000 7fb80137ab80 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdeeb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.747+0000 7fb80137ab80 1 --2- 172.21.15.47:0/1482913000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdeef810 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.747+0000 7fb80137ab80 1 -- 172.21.15.47:0/1482913000 >> 172.21.15.47:0/1482913000 conn(0x55e0cde17e60 msgr2=0x55e0cdee5cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:35.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.753+0000 7fb80137ab80 1 -- 172.21.15.47:0/1482913000 shutdown_connections 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.753+0000 7fb80137ab80 1 -- 172.21.15.47:0/1482913000 wait complete. 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.754+0000 7fb80137ab80 1 -- start start 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.755+0000 7fb80137ab80 1 Processor -- start 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.756+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdef7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.759+0000 7fb7f5081700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb7e00b2a50 0x7fb7e00b4f10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a9e90 tx=0x7fb7dc010fe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.758+0000 7fb7effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdef7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.759+0000 7fb7effff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdef7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:46134/0 (socket says 172.21.15.47:46134) 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.759+0000 7fb7effff700 1 -- 172.21.15.47:0/2854723293 learned_addr learned my addr 172.21.15.47:0/2854723293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.759+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdef4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.760+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdef4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.760+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdef7fb0 con 0x55e0cdeefd50 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.760+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdda3c30 con 0x55e0cdeebba0 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.760+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdef8130 con 0x55e0cdeeb280 2024-05-11T12:27:35.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.760+0000 7fb80137ab80 10 client.0 _ll_register_callbacks cb 0x55e0cddd5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.761+0000 7fb7effff700 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdef4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.762+0000 7fb7f4880700 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdef4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.764+0000 7fb7f4880700 1 -- 172.21.15.47:0/2854723293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 msgr2=0x55e0cdef7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.764+0000 7fb7f4880700 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdef7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.764+0000 7fb7f4880700 1 -- 172.21.15.47:0/2854723293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 msgr2=0x55e0cdef4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.764+0000 7fb7f4880700 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdef4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.765+0000 7fb7f4880700 1 -- 172.21.15.47:0/2854723293 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0cddd9690 con 0x55e0cdeebba0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.767+0000 7fb7f4880700 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdef4570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400bd30 tx=0x7fb7e4004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4474f5bf23911307 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.767+0000 7fb7cffff700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.773+0000 7fb7cffff700 1 -- 172.21.15.47:0/2854723293 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4028060 con 0x55e0cdeebba0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.774+0000 7fb7cffff700 1 -- 172.21.15.47:0/2854723293 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb7e400b2c0 con 0x55e0cdeebba0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.774+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7e400fe50 con 0x55e0cdeebba0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.775+0000 7fb7cffff700 1 -- 172.21.15.47:0/2854723293 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb7e4015040 con 0x55e0cdeebba0 2024-05-11T12:27:35.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.775+0000 7fb80137ab80 10 client.25001 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.775+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e0cdf02890 con 0x55e0cdeebba0 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.775+0000 7fb7cffff700 1 -- 172.21.15.47:0/2854723293 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7e400fe50 con 0x55e0cdeebba0 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.775+0000 7fb7cffff700 1 -- 172.21.15.47:0/2854723293 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb7e4022030 con 0x55e0cdeebba0 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.775+0000 7fb80137ab80 10 client.25001 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.776+0000 7fb80137ab80 10 client.25001 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.776+0000 7fb80137ab80 20 client.25001 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.776+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf04d60 0x55e0cdf07150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.776+0000 7fb80137ab80 4 client.25001 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.776+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0cde4b240 con 0x55e0cdf04d60 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.778+0000 7fb7effff700 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf04d60 0x55e0cdf07150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.798+0000 7fb7effff700 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf04d60 0x55e0cdf07150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c879f6f95ca62d1a server_cookie=9ac7ef2b0d308dbd in_seq=0 out_seq=0 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.798+0000 7fb7cffff700 10 client.25001 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.799+0000 7fb7cffff700 1 -- 172.21.15.47:0/2854723293 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e0cde4b240 con 0x55e0cdf04d60 2024-05-11T12:27:35.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.799+0000 7fb7cffff700 10 client.25001 handle_command_reply: tid=0 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.799+0000 7fb80137ab80 1 client.25001 shutdown 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 msgr2=0x55e0cdef4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdef4570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400bd30 tx=0x7fb7e4004320 comp rx=0 tx=0).stop 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 shutdown_connections 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf04d60 0x55e0cdf07150 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdef4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdef7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2854723293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdef4570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 >> 172.21.15.47:0/2854723293 conn(0x55e0cde17e60 msgr2=0x55e0cdeea090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb7cffff700 0 client.25001 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:35.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.800+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 shutdown_connections 2024-05-11T12:27:35.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.802+0000 7fb80137ab80 1 -- 172.21.15.47:0/2854723293 wait complete. 2024-05-11T12:27:35.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.803+0000 7fb80137ab80 20 client.25001 trim_cache size 0 max 16384 2024-05-11T12:27:35.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.842+0000 7fb80137ab80 1 Processor -- start 2024-05-11T12:27:35.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.844+0000 7fb80137ab80 1 -- start start 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.845+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdef4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.845+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdef51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.845+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.845+0000 7fb80137ab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d400b800 con 0x55e0cdeebba0 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.845+0000 7fb80137ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d400e080 con 0x55e0cdeeb280 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.845+0000 7fb7effff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdef4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.846+0000 7fb7effff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdef4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:60956/0 (socket says 172.21.15.47:60956) 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.846+0000 7fb7effff700 1 -- 172.21.15.47:0/2209498727 learned_addr learned my addr 172.21.15.47:0/2209498727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.846+0000 7fb80137ab80 1 -- 172.21.15.47:0/2209498727 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d400be70 con 0x55e0cdeefd50 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.846+0000 7fb7effff700 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.847+0000 7fb7f4880700 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdef51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.847+0000 7fb7effff700 1 -- 172.21.15.47:0/2209498727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 msgr2=0x55e0cdef6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.847+0000 7fb7effff700 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.847+0000 7fb7effff700 1 -- 172.21.15.47:0/2209498727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 msgr2=0x55e0cdef51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.847+0000 7fb7effff700 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdef51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.847+0000 7fb7effff700 1 -- 172.21.15.47:0/2209498727 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0cdf01460 con 0x55e0cdeeb280 2024-05-11T12:27:35.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.854+0000 7fb7effff700 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdef4ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0069630 tx=0x7fb7f00696d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8d19fd4b16aa1a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:35.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.858+0000 7fb7da7fc700 1 -- 172.21.15.47:0/2209498727 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f006a550 con 0x55e0cdeeb280 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.858+0000 7fb7da7fc700 1 -- 172.21.15.47:0/2209498727 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f006aba0 con 0x55e0cdeeb280 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.858+0000 7fb7da7fc700 1 -- 172.21.15.47:0/2209498727 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0079a50 con 0x55e0cdeeb280 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.861+0000 7fb80137ab80 1 -- 172.21.15.47:0/2209498727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 msgr2=0x55e0cdef4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.862+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdef4ae0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0069630 tx=0x7fb7f00696d0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.862+0000 7fb80137ab80 1 -- 172.21.15.47:0/2209498727 shutdown_connections 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.862+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeefd50 0x55e0cdef6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.862+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeeb280 0x55e0cdef4ae0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.862+0000 7fb80137ab80 1 --2- 172.21.15.47:0/2209498727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeebba0 0x55e0cdef51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.862+0000 7fb80137ab80 1 -- 172.21.15.47:0/2209498727 >> 172.21.15.47:0/2209498727 conn(0x55e0cde17e60 msgr2=0x55e0cdef2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.864+0000 7fb80137ab80 1 -- 172.21.15.47:0/2209498727 shutdown_connections 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.864+0000 7fb80137ab80 1 -- 172.21.15.47:0/2209498727 wait complete. 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.864+0000 7fb80137ab80 1 -- start start 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.865+0000 7fb80137ab80 1 Processor -- start 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.866+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdf06900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.866+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdf06e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.866+0000 7fb80137ab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdf07380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.866+0000 7fb80137ab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d400c540 con 0x55e0cdeefd50 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.866+0000 7fb80137ab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d400d9b0 con 0x55e0cdeebba0 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.866+0000 7fb80137ab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d400cc10 con 0x55e0cdeeb280 2024-05-11T12:27:35.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.866+0000 7fb80137ab80 10 client.0 _ll_register_callbacks cb 0x55e0cddd5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:35.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.868+0000 7fb7f4880700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdf06e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.868+0000 7fb7f4880700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdf06e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:60968/0 (socket says 172.21.15.47:60968) 2024-05-11T12:27:35.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.868+0000 7fb7f4880700 1 -- 172.21.15.47:0/510102101 learned_addr learned my addr 172.21.15.47:0/510102101 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.870+0000 7fb7effff700 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdf07380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.871+0000 7fb7effff700 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdf06900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.872+0000 7fb7f4880700 1 -- 172.21.15.47:0/510102101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 msgr2=0x55e0cdf06900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.872+0000 7fb7f4880700 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdf06900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.872+0000 7fb7f4880700 1 -- 172.21.15.47:0/510102101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 msgr2=0x55e0cdf07380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.872+0000 7fb7f4880700 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdf07380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.872+0000 7fb7f4880700 1 -- 172.21.15.47:0/510102101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0cdf6d270 con 0x55e0cdeebba0 2024-05-11T12:27:35.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.883+0000 7fb7f4880700 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdf06e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55e0cdee7ad0 tx=0x7fb7e400b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533d99f3959776de server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.883+0000 7fb7cffff700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:27:35.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.887+0000 7fb7cffff700 1 -- 172.21.15.47:0/510102101 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4028070 con 0x55e0cdeebba0 2024-05-11T12:27:35.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.887+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7f006aba0 con 0x55e0cdeebba0 2024-05-11T12:27:35.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.887+0000 7fb7cffff700 1 -- 172.21.15.47:0/510102101 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb7e400fcf0 con 0x55e0cdeebba0 2024-05-11T12:27:35.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.890+0000 7fb7cffff700 1 -- 172.21.15.47:0/510102101 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb7e403d050 con 0x55e0cdeebba0 2024-05-11T12:27:35.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.895+0000 7fb80137ab80 10 client.25007 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:35.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.895+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e0cdf6e540 con 0x55e0cdeebba0 2024-05-11T12:27:35.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.897+0000 7fb7cffff700 1 -- 172.21.15.47:0/510102101 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f006aba0 con 0x55e0cdeebba0 2024-05-11T12:27:35.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.897+0000 7fb7cffff700 1 -- 172.21.15.47:0/510102101 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb7e4022070 con 0x55e0cdeebba0 2024-05-11T12:27:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.899+0000 7fb80137ab80 10 client.25007 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.899+0000 7fb80137ab80 10 client.25007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.899+0000 7fb80137ab80 20 client.25007 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.899+0000 7fb80137ab80 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf6eb80 0x55e0cdf70fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.899+0000 7fb80137ab80 4 client.25007 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:27:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.899+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0cde452e0 con 0x55e0cdf6eb80 2024-05-11T12:27:35.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.900+0000 7fb7effff700 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf6eb80 0x55e0cdf70fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.904+0000 7fb7effff700 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf6eb80 0x55e0cdf70fa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdbcd2c87877297 server_cookie=e75e8aa24248b84a in_seq=0 out_seq=0 2024-05-11T12:27:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.904+0000 7fb7cffff700 10 client.25007 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.904+0000 7fb7cffff700 1 -- 172.21.15.47:0/510102101 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0cde452e0 con 0x55e0cdf6eb80 2024-05-11T12:27:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.904+0000 7fb7cffff700 10 client.25007 handle_command_reply: tid=0 2024-05-11T12:27:35.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.908+0000 7fb80137ab80 1 client.25007 shutdown 2024-05-11T12:27:35.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 msgr2=0x55e0cdf06e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdf06e40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55e0cdee7ad0 tx=0x7fb7e400b5a0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 shutdown_connections 2024-05-11T12:27:35.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e0cdf6eb80 0x55e0cdf70fa0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e0cdeefd50 0x55e0cdf07380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e0cdeeb280 0x55e0cdf06900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 --2- 172.21.15.47:0/510102101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e0cdeebba0 0x55e0cdf06e40 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55e0cdee7ad0 tx=0x7fb7e400b5a0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb7cffff700 0 client.25007 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:35.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.916+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 >> 172.21.15.47:0/510102101 conn(0x55e0cde17e60 msgr2=0x55e0cdeeebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:35.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.917+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 shutdown_connections 2024-05-11T12:27:35.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.917+0000 7fb80137ab80 1 -- 172.21.15.47:0/510102101 wait complete. 2024-05-11T12:27:35.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.917+0000 7fb80137ab80 20 client.25007 trim_cache size 0 max 16384 2024-05-11T12:27:35.920 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:27:35.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb7e00b2a50 msgr2=0x7fb7e00b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb7e00b2a50 0x7fb7e00b4f10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc0a9e90 tx=0x7fb7dc010fe0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80b7af0 msgr2=0x7fb7e80af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:35.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80af140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00b850 tx=0x7fb7dc00b930 comp rx=0 tx=0).stop 2024-05-11T12:27:35.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 shutdown_connections 2024-05-11T12:27:35.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb7e00b2a50 0x7fb7e00b4f10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb7e80b7af0 0x7fb7e80af140 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb7e80a4dd0 0x7fb7e80b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 --2- 172.21.15.47:0/1136925305 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb7e80a56f0 0x7fb7e80aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:35.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.920+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 >> 172.21.15.47:0/1136925305 conn(0x7fb7e801a690 msgr2=0x7fb7e80a1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:35.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.924+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 shutdown_connections 2024-05-11T12:27:35.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:35.926+0000 7fb7f5882700 1 -- 172.21.15.47:0/1136925305 wait complete. 2024-05-11T12:27:37.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:36 smithi047 ceph-mon[30375]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.72k op/s 2024-05-11T12:27:37.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:27:37.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:27:37.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:36 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:27:37.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:36 smithi050 ceph-mon[34072]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.72k op/s 2024-05-11T12:27:37.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:27:37.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:27:37.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:36 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:27:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:36 smithi064 ceph-mon[34537]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 726 MiB/s wr, 1.72k op/s 2024-05-11T12:27:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:27:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:27:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:36 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:27:38.788 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:27:38.789 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:27:39.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:38 smithi050 ceph-mon[34072]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 2.15k op/s 2024-05-11T12:27:39.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:38 smithi047 ceph-mon[30375]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 2.15k op/s 2024-05-11T12:27:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:38 smithi064 ceph-mon[34537]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 888 MiB/s wr, 2.15k op/s 2024-05-11T12:27:39.842 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:27:41.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:40 smithi050 ceph-mon[34072]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 746 MiB/s wr, 2.01k op/s 2024-05-11T12:27:41.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:40 smithi047 ceph-mon[30375]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 746 MiB/s wr, 2.01k op/s 2024-05-11T12:27:41.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:40 smithi064 ceph-mon[34537]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 746 MiB/s wr, 2.01k op/s 2024-05-11T12:27:43.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:42 smithi047 ceph-mon[30375]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 839 MiB/s wr, 2.10k op/s 2024-05-11T12:27:43.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:42 smithi064 ceph-mon[34537]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 839 MiB/s wr, 2.10k op/s 2024-05-11T12:27:43.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:42 smithi050 ceph-mon[34072]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 839 MiB/s wr, 2.10k op/s 2024-05-11T12:27:44.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.659+0000 7ff30228e700 1 -- 172.21.15.47:0/1181455764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc102450 msgr2=0x7ff2fc1028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.659+0000 7ff30228e700 1 --2- 172.21.15.47:0/1181455764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc102450 0x7ff2fc1028d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009990 tx=0x7ff2ec009ca0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.660+0000 7ff30228e700 1 -- 172.21.15.47:0/1181455764 shutdown_connections 2024-05-11T12:27:44.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.660+0000 7ff30228e700 1 --2- 172.21.15.47:0/1181455764 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc102e10 0x7ff2fc10f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.660+0000 7ff30228e700 1 --2- 172.21.15.47:0/1181455764 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc108530 0x7ff2fc108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.660+0000 7ff30228e700 1 --2- 172.21.15.47:0/1181455764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc102450 0x7ff2fc1028d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.660+0000 7ff30228e700 1 -- 172.21.15.47:0/1181455764 >> 172.21.15.47:0/1181455764 conn(0x7ff2fc079400 msgr2=0x7ff2fc079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:44.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.668+0000 7ff30228e700 1 -- 172.21.15.47:0/1181455764 shutdown_connections 2024-05-11T12:27:44.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.669+0000 7ff30228e700 1 -- 172.21.15.47:0/1181455764 wait complete. 2024-05-11T12:27:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.674+0000 7ff30228e700 1 Processor -- start 2024-05-11T12:27:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.675+0000 7ff30228e700 1 -- start start 2024-05-11T12:27:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.677+0000 7ff30228e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc102450 0x7ff2fc19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.677+0000 7ff30228e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc102e10 0x7ff2fc19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.677+0000 7ff30228e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc108530 0x7ff2fc197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.678+0000 7ff30228e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc111680 con 0x7ff2fc102e10 2024-05-11T12:27:44.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.678+0000 7ff30228e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc111500 con 0x7ff2fc108530 2024-05-11T12:27:44.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.678+0000 7ff30228e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc111800 con 0x7ff2fc102450 2024-05-11T12:27:44.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.678+0000 7ff30082b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc108530 0x7ff2fc197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.678+0000 7ff30082b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc108530 0x7ff2fc197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:50646/0 (socket says 172.21.15.47:50646) 2024-05-11T12:27:44.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.678+0000 7ff30082b700 1 -- 172.21.15.47:0/1466231415 learned_addr learned my addr 172.21.15.47:0/1466231415 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:44.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.683+0000 7ff30082b700 1 -- 172.21.15.47:0/1466231415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc102450 msgr2=0x7ff2fc19d8c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:27:44.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.683+0000 7ff30082b700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc102450 0x7ff2fc19d8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.683+0000 7ff30082b700 1 -- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc102e10 msgr2=0x7ff2fc19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.683+0000 7ff30082b700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc102e10 0x7ff2fc19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.683+0000 7ff30082b700 1 -- 172.21.15.47:0/1466231415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2fc1981e0 con 0x7ff2fc108530 2024-05-11T12:27:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.687+0000 7ff30082b700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc108530 0x7ff2fc197a80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400b550 tx=0x7ff2e400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.689+0000 7ff2f97fa700 1 -- 172.21.15.47:0/1466231415 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4015020 con 0x7ff2fc108530 2024-05-11T12:27:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.689+0000 7ff2f97fa700 1 -- 172.21.15.47:0/1466231415 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e4011ce0 con 0x7ff2fc108530 2024-05-11T12:27:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.689+0000 7ff2f97fa700 1 -- 172.21.15.47:0/1466231415 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4010d40 con 0x7ff2fc108530 2024-05-11T12:27:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.689+0000 7ff30228e700 1 -- 172.21.15.47:0/1466231415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2fc1984d0 con 0x7ff2fc108530 2024-05-11T12:27:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.689+0000 7ff30228e700 1 -- 172.21.15.47:0/1466231415 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2fc1a4790 con 0x7ff2fc108530 2024-05-11T12:27:44.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.697+0000 7ff30cb24b80 1 Processor -- start 2024-05-11T12:27:44.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.697+0000 7ff30cb24b80 1 -- start start 2024-05-11T12:27:44.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.697+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b0d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.704+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b0d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.704+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b0d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53894/0 (socket says 172.21.15.47:53894) 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.704+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1898257329 learned_addr learned my addr 172.21.15.47:0/1898257329 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.704+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b0b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.704+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b05570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.705+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf17a5a9c0 con 0x55bf17b0d190 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.705+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bf179bfe20 con 0x55bf17b0b890 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.705+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf17a56980 con 0x55bf17b0aed0 2024-05-11T12:27:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.705+0000 7ff2f97fa700 1 -- 172.21.15.47:0/1466231415 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff2e4018020 con 0x7ff2fc108530 2024-05-11T12:27:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.706+0000 7ff2f97fa700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2e80b2aa0 0x7ff2e80b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.707+0000 7ff2f97fa700 1 -- 172.21.15.47:0/1466231415 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff2e40e1a30 con 0x7ff2fc108530 2024-05-11T12:27:44.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.709+0000 7ff30082b700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2e80b2aa0 0x7ff2e80b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.711+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b05570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.715+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1898257329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 msgr2=0x55bf17b0b350 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:27:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.715+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b0b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.715+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1898257329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 msgr2=0x55bf17b05570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.715+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b05570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.716+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1898257329 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf179a7950 con 0x55bf17b0d190 2024-05-11T12:27:44.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.721+0000 7ff30082b700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2e80b2aa0 0x7ff2e80b4f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40a8c20 tx=0x7ff2e4004860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:44.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.727+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b0d570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005cb0 tx=0x7ff2ec005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467788600957daa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:44.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.727+0000 7ff2f1ffb700 1 -- 172.21.15.47:0/1898257329 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec01a040 con 0x55bf17b0d190 2024-05-11T12:27:44.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.728+0000 7ff2f1ffb700 1 -- 172.21.15.47:0/1898257329 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ec01b040 con 0x55bf17b0d190 2024-05-11T12:27:44.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.728+0000 7ff2f1ffb700 1 -- 172.21.15.47:0/1898257329 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec005400 con 0x55bf17b0d190 2024-05-11T12:27:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.729+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 msgr2=0x55bf17b0d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.729+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b0d570 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005cb0 tx=0x7ff2ec005c00 comp rx=0 tx=0).stop 2024-05-11T12:27:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.730+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 shutdown_connections 2024-05-11T12:27:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.730+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b05570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.730+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b0d570 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.730+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1898257329 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b0b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.730+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 >> 172.21.15.47:0/1898257329 conn(0x55bf17a34bf0 msgr2=0x55bf17b03300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.733+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 shutdown_connections 2024-05-11T12:27:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.733+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1898257329 wait complete. 2024-05-11T12:27:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.734+0000 7ff30cb24b80 1 -- start start 2024-05-11T12:27:44.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.735+0000 7ff30cb24b80 1 Processor -- start 2024-05-11T12:27:44.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.735+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b28d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.736+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.736+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b23190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.736+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55bf17b05c00 con 0x55bf17b0d190 2024-05-11T12:27:44.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.736+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55bf17a13f50 con 0x55bf17b0b890 2024-05-11T12:27:44.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.736+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55bf17b05d80 con 0x55bf17b0aed0 2024-05-11T12:27:44.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.737+0000 7ff30cb24b80 10 client.0 _ll_register_callbacks cb 0x55bf179f29d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:44.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.738+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b28d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.738+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b28d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:34896/0 (socket says 172.21.15.47:34896) 2024-05-11T12:27:44.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.738+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/808462424 learned_addr learned my addr 172.21.15.47:0/808462424 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:44.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.738+0000 7ff2fbfff700 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.739+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b23190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.741+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 msgr2=0x55bf17b292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.741+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.741+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/808462424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 msgr2=0x55bf17b23190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.741+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b23190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.741+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/808462424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf17a117e0 con 0x55bf17b0aed0 2024-05-11T12:27:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.743+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b28d70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec003e40 tx=0x7ff2ec00d790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b1db393e97cff06 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.744+0000 7ff2db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:27:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.748+0000 7ff2db7fe700 1 -- 172.21.15.47:0/808462424 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec01a040 con 0x55bf17b0aed0 2024-05-11T12:27:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.748+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ec01b040 con 0x55bf17b0aed0 2024-05-11T12:27:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.748+0000 7ff2db7fe700 1 -- 172.21.15.47:0/808462424 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff2ec012050 con 0x55bf17b0aed0 2024-05-11T12:27:44.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.753+0000 7ff2db7fe700 1 -- 172.21.15.47:0/808462424 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec0056b0 con 0x55bf17b0aed0 2024-05-11T12:27:44.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.753+0000 7ff30cb24b80 10 client.25057 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:44.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.753+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bf17b05c00 con 0x55bf17b0aed0 2024-05-11T12:27:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff2db7fe700 1 -- 172.21.15.47:0/808462424 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ec01b040 con 0x55bf17b0aed0 2024-05-11T12:27:44.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff2db7fe700 1 -- 172.21.15.47:0/808462424 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff2ec00f640 con 0x55bf17b0aed0 2024-05-11T12:27:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff30cb24b80 10 client.25057 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff30cb24b80 10 client.25057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff30cb24b80 20 client.25057 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b32db0 0x55bf17b351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff30cb24b80 4 client.25057 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:27:44.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.756+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf17a683a0 con 0x55bf17b32db0 2024-05-11T12:27:44.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.758+0000 7ff2fbfff700 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b32db0 0x55bf17b351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.764+0000 7ff2fbfff700 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b32db0 0x55bf17b351f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b823b65c00a694aa server_cookie=1673bd1d373b1bb6 in_seq=0 out_seq=0 2024-05-11T12:27:44.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.765+0000 7ff2db7fe700 10 client.25057 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:44.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.770+0000 7ff2db7fe700 1 -- 172.21.15.47:0/808462424 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bf17a683a0 con 0x55bf17b32db0 2024-05-11T12:27:44.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.770+0000 7ff2db7fe700 10 client.25057 handle_command_reply: tid=0 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 client.25057 shutdown 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 msgr2=0x55bf17b28d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b28d70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec003e40 tx=0x7ff2ec00d790 comp rx=0 tx=0).stop 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 shutdown_connections 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b32db0 0x55bf17b351f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0d190 0x55bf17b23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0aed0 0x55bf17b28d70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/808462424 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0b890 0x55bf17b292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 >> 172.21.15.47:0/808462424 conn(0x55bf17a34bf0 msgr2=0x55bf17afd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:44.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.771+0000 7ff2db7fe700 0 client.25057 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.775+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 shutdown_connections 2024-05-11T12:27:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.778+0000 7ff30cb24b80 1 -- 172.21.15.47:0/808462424 wait complete. 2024-05-11T12:27:44.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.778+0000 7ff30cb24b80 20 client.25057 trim_cache size 0 max 16384 2024-05-11T12:27:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 Processor -- start 2024-05-11T12:27:44.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 -- start start 2024-05-11T12:27:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 0x55bf17b07b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0b890 0x55bf17afd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0d190 0x55bf17b3aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1a9fc0 con 0x55bf17b0aed0 2024-05-11T12:27:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1ac840 con 0x55bf17b0d190 2024-05-11T12:27:44.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.854+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1aa630 con 0x55bf17b0b890 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.855+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0d190 0x55bf17b3aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.855+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0d190 0x55bf17b3aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:50676/0 (socket says 172.21.15.47:50676) 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.855+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1697850091 learned_addr learned my addr 172.21.15.47:0/1697850091 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.856+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1697850091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0b890 msgr2=0x55bf17afd7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.856+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/1697850091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0b890 0x55bf17afd7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.856+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1697850091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 msgr2=0x55bf17b07b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.856+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/1697850091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 0x55bf17b07b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.856+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/1697850091 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf17ae5b70 con 0x55bf17b0d190 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.856+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/1697850091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0d190 0x55bf17b3aac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec015920 tx=0x7ff2ec005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=202643d0dfc3f00b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.857+0000 7ff2f17fa700 1 -- 172.21.15.47:0/1697850091 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec026020 con 0x55bf17b0d190 2024-05-11T12:27:44.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.857+0000 7ff2f17fa700 1 -- 172.21.15.47:0/1697850091 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ec01b020 con 0x55bf17b0d190 2024-05-11T12:27:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.858+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1697850091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0d190 msgr2=0x55bf17b3aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.858+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1697850091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0d190 0x55bf17b3aac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec015920 tx=0x7ff2ec005a00 comp rx=0 tx=0).stop 2024-05-11T12:27:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.858+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1697850091 shutdown_connections 2024-05-11T12:27:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.858+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1697850091 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b0d190 0x55bf17b3aac0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.858+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1697850091 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 0x55bf17b07b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.858+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/1697850091 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b0b890 0x55bf17afd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.858+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1697850091 >> 172.21.15.47:0/1697850091 conn(0x55bf17a34bf0 msgr2=0x55bf17afd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:44.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.864+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1697850091 shutdown_connections 2024-05-11T12:27:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.864+0000 7ff30cb24b80 1 -- 172.21.15.47:0/1697850091 wait complete. 2024-05-11T12:27:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.864+0000 7ff30cb24b80 1 -- start start 2024-05-11T12:27:44.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.865+0000 7ff30cb24b80 1 Processor -- start 2024-05-11T12:27:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.866+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 0x55bf17b2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.866+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b2bfe0 0x55bf17b2c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.866+0000 7ff30cb24b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b9a660 0x55bf17b9ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.866+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1aad00 con 0x55bf17b0aed0 2024-05-11T12:27:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.866+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1ac170 con 0x55bf17b2bfe0 2024-05-11T12:27:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.866+0000 7ff30cb24b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff2fc1ab3d0 con 0x55bf17b9a660 2024-05-11T12:27:44.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.866+0000 7ff30cb24b80 10 client.0 _ll_register_callbacks cb 0x55bf179f29d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:44.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.868+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b9a660 0x55bf17b9ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.868+0000 7ff2fb7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b9a660 0x55bf17b9ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:34910/0 (socket says 172.21.15.47:34910) 2024-05-11T12:27:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.868+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/135997555 learned_addr learned my addr 172.21.15.47:0/135997555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.868+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 0x55bf17b2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.874+0000 7ff2fbfff700 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b2bfe0 0x55bf17b2c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.875+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b2bfe0 msgr2=0x55bf17b2c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.875+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b2bfe0 0x55bf17b2c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.875+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/135997555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 msgr2=0x55bf17b2baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.875+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 0x55bf17b2baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.875+0000 7ff2fb7fe700 1 -- 172.21.15.47:0/135997555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf17b0a180 con 0x55bf17b9a660 2024-05-11T12:27:44.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.878+0000 7ff2fb7fe700 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b9a660 0x55bf17b9ca50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec003760 tx=0x7ff2ec00d790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4284622ef883b96a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:44.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.878+0000 7ff2db7fe700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:27:44.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.881+0000 7ff2db7fe700 1 -- 172.21.15.47:0/135997555 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec03f060 con 0x55bf17b9a660 2024-05-11T12:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.881+0000 7ff2db7fe700 1 -- 172.21.15.47:0/135997555 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff2ec0156e0 con 0x55bf17b9a660 2024-05-11T12:27:44.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.888+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bf17a62440 con 0x55bf17b9a660 2024-05-11T12:27:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.894+0000 7ff2db7fe700 1 -- 172.21.15.47:0/135997555 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff2ec026070 con 0x55bf17b9a660 2024-05-11T12:27:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.894+0000 7ff30cb24b80 10 client.25060 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:44.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.894+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bf17b9e0f0 con 0x55bf17b9a660 2024-05-11T12:27:44.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff2db7fe700 1 -- 172.21.15.47:0/135997555 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bf17a62440 con 0x55bf17b9a660 2024-05-11T12:27:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff2db7fe700 1 -- 172.21.15.47:0/135997555 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff2ec043420 con 0x55bf17b9a660 2024-05-11T12:27:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff30cb24b80 10 client.25060 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff30cb24b80 10 client.25060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff30cb24b80 20 client.25060 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b9e5b0 0x55bf17ba09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff30cb24b80 4 client.25060 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:27:44.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.900+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bf178da2d0 con 0x55bf17b9e5b0 2024-05-11T12:27:44.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.903+0000 7ff2fbfff700 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b9e5b0 0x55bf17ba09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.906+0000 7ff2fbfff700 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b9e5b0 0x55bf17ba09d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3fbbb47179ce174 server_cookie=4c947c0c1e67549e in_seq=0 out_seq=0 2024-05-11T12:27:44.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.906+0000 7ff2db7fe700 10 client.25060 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.931+0000 7ff2db7fe700 1 -- 172.21.15.47:0/135997555 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bf178da2d0 con 0x55bf17b9e5b0 2024-05-11T12:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.931+0000 7ff2db7fe700 10 client.25060 handle_command_reply: tid=0 2024-05-11T12:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.931+0000 7ff30cb24b80 1 client.25060 shutdown 2024-05-11T12:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b9a660 msgr2=0x55bf17b9ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b9a660 0x55bf17b9ca50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec003760 tx=0x7ff2ec00d790 comp rx=0 tx=0).stop 2024-05-11T12:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 shutdown_connections 2024-05-11T12:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55bf17b9e5b0 0x55bf17ba09d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55bf17b9a660 0x55bf17b9ca50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55bf17b0aed0 0x55bf17b2baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff30cb24b80 1 --2- 172.21.15.47:0/135997555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55bf17b2bfe0 0x55bf17b2c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.932+0000 7ff2db7fe700 0 client.25060 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:44.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.933+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 >> 172.21.15.47:0/135997555 conn(0x55bf17a34bf0 msgr2=0x55bf17afd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.933+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 shutdown_connections 2024-05-11T12:27:44.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.933+0000 7ff30cb24b80 1 -- 172.21.15.47:0/135997555 wait complete. 2024-05-11T12:27:44.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.935+0000 7ff30cb24b80 20 client.25060 trim_cache size 0 max 16384 2024-05-11T12:27:44.936 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:27:44.937 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:27:44.937 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "41d7f842-7a29-47af-8ae4-1d6c1e8befa6", 2024-05-11T12:27:44.937 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:27:44.937 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:27:44.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 -- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2e80b2aa0 msgr2=0x7ff2e80b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2e80b2aa0 0x7ff2e80b4f60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff2e40a8c20 tx=0x7ff2e4004860 comp rx=0 tx=0).stop 2024-05-11T12:27:44.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 -- 172.21.15.47:0/1466231415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc108530 msgr2=0x7ff2fc197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:44.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc108530 0x7ff2fc197a80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400b550 tx=0x7ff2e400b910 comp rx=0 tx=0).stop 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 -- 172.21.15.47:0/1466231415 shutdown_connections 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff2e80b2aa0 0x7ff2e80b4f60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff2fc108530 0x7ff2fc197a80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff2fc102450 0x7ff2fc19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 --2- 172.21.15.47:0/1466231415 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2fc102e10 0x7ff2fc19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.938+0000 7ff2f2ffd700 1 -- 172.21.15.47:0/1466231415 >> 172.21.15.47:0/1466231415 conn(0x7ff2fc079400 msgr2=0x7ff2fc1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.939+0000 7ff2f2ffd700 1 -- 172.21.15.47:0/1466231415 shutdown_connections 2024-05-11T12:27:44.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:44.939+0000 7ff2f2ffd700 1 -- 172.21.15.47:0/1466231415 wait complete. 2024-05-11T12:27:45.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:44 smithi047 ceph-mon[30375]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 850 MiB/s wr, 2.00k op/s 2024-05-11T12:27:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:44 smithi064 ceph-mon[34537]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 850 MiB/s wr, 2.00k op/s 2024-05-11T12:27:45.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:44 smithi050 ceph-mon[34072]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 850 MiB/s wr, 2.00k op/s 2024-05-11T12:27:46.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:45 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:27:46.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:45 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:46.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:45 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:27:46.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:45 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:46.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:45 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:27:46.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:45 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:46.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:45 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:27:46.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:45 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:46.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:45 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:27:46.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:45 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:46.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:45 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:27:46.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:45 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:27:46.813 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:27:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:46 smithi047 ceph-mon[30375]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 MiB/s wr, 1.66k op/s 2024-05-11T12:27:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:46 smithi064 ceph-mon[34537]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 MiB/s wr, 1.66k op/s 2024-05-11T12:27:47.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:46 smithi050 ceph-mon[34072]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 736 MiB/s wr, 1.66k op/s 2024-05-11T12:27:47.570 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:27:49.306 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:49 smithi047 ceph-mon[30375]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 904 MiB/s wr, 1.96k op/s 2024-05-11T12:27:49.307 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:49 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:27:49.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:49 smithi064 ceph-mon[34537]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 904 MiB/s wr, 1.96k op/s 2024-05-11T12:27:49.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:49 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:27:49.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:49 smithi050 ceph-mon[34072]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 904 MiB/s wr, 1.96k op/s 2024-05-11T12:27:49.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:49 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:27:51.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:51 smithi047 ceph-mon[30375]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 772 MiB/s wr, 1.54k op/s 2024-05-11T12:27:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:51 smithi064 ceph-mon[34537]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 772 MiB/s wr, 1.54k op/s 2024-05-11T12:27:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:51 smithi050 ceph-mon[34072]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 772 MiB/s wr, 1.54k op/s 2024-05-11T12:27:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.526+0000 7f1c9759e700 1 -- 172.21.15.47:0/3287957101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102fb0 msgr2=0x7f1c9810f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.526+0000 7f1c9759e700 1 --2- 172.21.15.47:0/3287957101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102fb0 0x7f1c9810f260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b0a0 tx=0x7f1c9000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.527+0000 7f1c9759e700 1 -- 172.21.15.47:0/3287957101 shutdown_connections 2024-05-11T12:27:52.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.527+0000 7f1c9759e700 1 --2- 172.21.15.47:0/3287957101 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102fb0 0x7f1c9810f260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.527+0000 7f1c9759e700 1 --2- 172.21.15.47:0/3287957101 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c98108420 0x7f1c98108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.527+0000 7f1c9759e700 1 --2- 172.21.15.47:0/3287957101 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98102610 0x7f1c98102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.527+0000 7f1c9759e700 1 -- 172.21.15.47:0/3287957101 >> 172.21.15.47:0/3287957101 conn(0x7f1c98079290 msgr2=0x7f1c980796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.528+0000 7f1c9759e700 1 -- 172.21.15.47:0/3287957101 shutdown_connections 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.528+0000 7f1c9759e700 1 -- 172.21.15.47:0/3287957101 wait complete. 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.530+0000 7f1c9759e700 1 Processor -- start 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.530+0000 7f1c9759e700 1 -- start start 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.531+0000 7f1c9759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102610 0x7f1c98197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.531+0000 7f1c9759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98102fb0 0x7f1c981981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.531+0000 7f1c9759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c98108420 0x7f1c9819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.531+0000 7f1c9759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1c981125c0 con 0x7f1c98102610 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.531+0000 7f1c9759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98112440 con 0x7f1c98102fb0 2024-05-11T12:27:52.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.531+0000 7f1c9759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98112740 con 0x7f1c98108420 2024-05-11T12:27:52.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.532+0000 7f1c9659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102610 0x7f1c98197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.532+0000 7f1c9659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102610 0x7f1c98197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59610/0 (socket says 172.21.15.47:59610) 2024-05-11T12:27:52.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.533+0000 7f1c9659c700 1 -- 172.21.15.47:0/2476782583 learned_addr learned my addr 172.21.15.47:0/2476782583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:52.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.533+0000 7f1c8dd9b700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98102fb0 0x7f1c981981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.533+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c98108420 0x7f1c9819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.534+0000 7f1c96d9d700 1 -- 172.21.15.47:0/2476782583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98102fb0 msgr2=0x7f1c981981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.534+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98102fb0 0x7f1c981981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.534+0000 7f1c96d9d700 1 -- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102610 msgr2=0x7f1c98197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.534+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102610 0x7f1c98197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.534+0000 7f1c96d9d700 1 -- 172.21.15.47:0/2476782583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9819c700 con 0x7f1c98108420 2024-05-11T12:27:52.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.543+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c98108420 0x7f1c9819c1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b070 tx=0x7f1c90004f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:52.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.559+0000 7f1c8ffff700 1 -- 172.21.15.47:0/2476782583 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c90020070 con 0x7f1c98108420 2024-05-11T12:27:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.559+0000 7f1c8ffff700 1 -- 172.21.15.47:0/2476782583 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c90012bc0 con 0x7f1c98108420 2024-05-11T12:27:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.559+0000 7f1c8ffff700 1 -- 172.21.15.47:0/2476782583 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9001c840 con 0x7f1c98108420 2024-05-11T12:27:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.561+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9819c990 con 0x7f1c98108420 2024-05-11T12:27:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.561+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9819ce80 con 0x7f1c98108420 2024-05-11T12:27:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.569+0000 7f1c8ffff700 1 -- 172.21.15.47:0/2476782583 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1c90023070 con 0x7f1c98108420 2024-05-11T12:27:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.571+0000 7f1c8ffff700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c840b2a80 0x7f1c840b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.571+0000 7f1c8ffff700 1 -- 172.21.15.47:0/2476782583 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1c9000f030 con 0x7f1c98108420 2024-05-11T12:27:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.574+0000 7f1ca7450b80 1 Processor -- start 2024-05-11T12:27:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.574+0000 7f1ca7450b80 1 -- start start 2024-05-11T12:27:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.574+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49280 0x55ee61e49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.574+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e4d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.574+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e4dd50 0x55ee61e4e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.575+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ee61d9b860 con 0x55ee61e49ba0 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.575+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ee61d3a7c0 con 0x55ee61e4dd50 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.575+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee61d97820 con 0x55ee61e49280 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.575+0000 7f1c9659c700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c840b2a80 0x7f1c840b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.575+0000 7f1c8dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e4d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.575+0000 7f1c8dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e4d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59622/0 (socket says 172.21.15.47:59622) 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.575+0000 7f1c8dd9b700 1 -- 172.21.15.47:0/3495526508 learned_addr learned my addr 172.21.15.47:0/3495526508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.576+0000 7f1c8dd9b700 1 -- 172.21.15.47:0/3495526508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49280 msgr2=0x55ee61e49660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.576+0000 7f1c8dd9b700 1 --2- 172.21.15.47:0/3495526508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49280 0x55ee61e49660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.576+0000 7f1c8dd9b700 1 -- 172.21.15.47:0/3495526508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e4dd50 msgr2=0x55ee61e4e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.576+0000 7f1c8dd9b700 1 --2- 172.21.15.47:0/3495526508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e4dd50 0x55ee61e4e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.576+0000 7f1c8dd9b700 1 -- 172.21.15.47:0/3495526508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee61ce88d0 con 0x55ee61e49ba0 2024-05-11T12:27:52.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.577+0000 7f1c8dd9b700 1 --2- 172.21.15.47:0/3495526508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e4d810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009d20 tx=0x7f1c88011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c35991c54d99f64 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.578+0000 7f1c8cd99700 1 -- 172.21.15.47:0/3495526508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88004a30 con 0x55ee61e49ba0 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.578+0000 7f1c8cd99700 1 -- 172.21.15.47:0/3495526508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c880054d0 con 0x55ee61e49ba0 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.578+0000 7f1c8cd99700 1 -- 172.21.15.47:0/3495526508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c88019880 con 0x55ee61e49ba0 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.579+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3495526508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 msgr2=0x55ee61e4d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.579+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3495526508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e4d810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009d20 tx=0x7f1c88011720 comp rx=0 tx=0).stop 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.580+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3495526508 shutdown_connections 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.580+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3495526508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e4dd50 0x55ee61e4e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.580+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3495526508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49280 0x55ee61e49660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.580+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3495526508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e4d810 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.580+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3495526508 >> 172.21.15.47:0/3495526508 conn(0x55ee61d75e60 msgr2=0x55ee61e43cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.581+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3495526508 shutdown_connections 2024-05-11T12:27:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.581+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3495526508 wait complete. 2024-05-11T12:27:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.582+0000 7f1ca7450b80 1 -- start start 2024-05-11T12:27:52.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.582+0000 7f1ca7450b80 1 Processor -- start 2024-05-11T12:27:52.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.583+0000 7f1c9659c700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c840b2a80 0x7f1c840b4f40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009d20 tx=0x7f1c80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:52.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.584+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.584+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e52690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.584+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e4dd50 0x55ee61e52bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.584+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ee61e55fb0 con 0x55ee61e49ba0 2024-05-11T12:27:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.584+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ee61d01c30 con 0x55ee61e49280 2024-05-11T12:27:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.584+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ee61e56130 con 0x55ee61e4dd50 2024-05-11T12:27:52.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.584+0000 7f1ca7450b80 10 client.0 _ll_register_callbacks cb 0x55ee61d33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:52.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.586+0000 7f1c9659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.586+0000 7f1c9659c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:40816/0 (socket says 172.21.15.47:40816) 2024-05-11T12:27:52.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.586+0000 7f1c9659c700 1 -- 172.21.15.47:0/4261766256 learned_addr learned my addr 172.21.15.47:0/4261766256 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:52.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.586+0000 7f1c8dd9b700 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e52690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.586+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e4dd50 0x55ee61e52bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.588+0000 7f1c9659c700 1 -- 172.21.15.47:0/4261766256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e4dd50 msgr2=0x55ee61e52bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:52.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.588+0000 7f1c9659c700 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e4dd50 0x55ee61e52bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.588+0000 7f1c9659c700 1 -- 172.21.15.47:0/4261766256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 msgr2=0x55ee61e52690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:52.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.588+0000 7f1c9659c700 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e52690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.588+0000 7f1c9659c700 1 -- 172.21.15.47:0/4261766256 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee61d37690 con 0x55ee61e49280 2024-05-11T12:27:52.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.595+0000 7f1c9659c700 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e552f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000ebe0 tx=0x7f1c800047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e9717cd48dcc396 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.599+0000 7f1c7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:27:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.599+0000 7f1c7dffb700 1 -- 172.21.15.47:0/4261766256 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000fa60 con 0x55ee61e49280 2024-05-11T12:27:52.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.599+0000 7f1c7dffb700 1 -- 172.21.15.47:0/4261766256 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1c8000fc00 con 0x55ee61e49280 2024-05-11T12:27:52.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.603+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c880054d0 con 0x55ee61e49280 2024-05-11T12:27:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.608+0000 7f1c7dffb700 1 -- 172.21.15.47:0/4261766256 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1c8000fe00 con 0x55ee61e49280 2024-05-11T12:27:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.608+0000 7f1ca7450b80 10 client.25031 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:52.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.608+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ee61e609e0 con 0x55ee61e49280 2024-05-11T12:27:52.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.615+0000 7f1c7dffb700 1 -- 172.21.15.47:0/4261766256 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c880054d0 con 0x55ee61e49280 2024-05-11T12:27:52.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.616+0000 7f1c7dffb700 1 -- 172.21.15.47:0/4261766256 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1c8001e620 con 0x55ee61e49280 2024-05-11T12:27:52.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.617+0000 7f1ca7450b80 10 client.25031 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:52.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.617+0000 7f1ca7450b80 10 client.25031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:52.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.617+0000 7f1ca7450b80 20 client.25031 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:52.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.618+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61e633d0 0x55ee61e65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.618+0000 7f1ca7450b80 4 client.25031 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:27:52.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.618+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee61da9240 con 0x55ee61e633d0 2024-05-11T12:27:52.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.622+0000 7f1c8dd9b700 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61e633d0 0x55ee61e65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.643+0000 7f1c8dd9b700 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61e633d0 0x55ee61e65830 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4306680fa74fe05 server_cookie=9515d096c20df51e in_seq=0 out_seq=0 2024-05-11T12:27:52.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.644+0000 7f1c7dffb700 10 client.25031 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:52.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.649+0000 7f1c7dffb700 1 -- 172.21.15.47:0/4261766256 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ee61da9240 con 0x55ee61e633d0 2024-05-11T12:27:52.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.650+0000 7f1c7dffb700 10 client.25031 handle_command_reply: tid=0 2024-05-11T12:27:52.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.650+0000 7f1ca7450b80 1 client.25031 shutdown 2024-05-11T12:27:52.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 msgr2=0x55ee61e552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e552f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000ebe0 tx=0x7f1c800047c0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 shutdown_connections 2024-05-11T12:27:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61e633d0 0x55ee61e65830 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e4dd50 0x55ee61e52bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e552f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/4261766256 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e49ba0 0x55ee61e52690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 >> 172.21.15.47:0/4261766256 conn(0x55ee61d75e60 msgr2=0x55ee61e48090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.651+0000 7f1c7dffb700 0 client.25031 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.652+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 shutdown_connections 2024-05-11T12:27:52.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.652+0000 7f1ca7450b80 1 -- 172.21.15.47:0/4261766256 wait complete. 2024-05-11T12:27:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.652+0000 7f1ca7450b80 20 client.25031 trim_cache size 0 max 16384 2024-05-11T12:27:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.720+0000 7f1ca7450b80 1 Processor -- start 2024-05-11T12:27:52.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.720+0000 7f1ca7450b80 1 -- start start 2024-05-11T12:27:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.721+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e48920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.721+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 0x55ee61e500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.721+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e4dd50 0x55ee61e54ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.722+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1c980723e0 con 0x55ee61e4dd50 2024-05-11T12:27:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.722+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98074c60 con 0x55ee61e49280 2024-05-11T12:27:52.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.722+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98072a50 con 0x55ee61e49ba0 2024-05-11T12:27:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.723+0000 7f1c96d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e4dd50 0x55ee61e54ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.723+0000 7f1c96d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e4dd50 0x55ee61e54ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59646/0 (socket says 172.21.15.47:59646) 2024-05-11T12:27:52.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.723+0000 7f1c96d9d700 1 -- 172.21.15.47:0/3641194258 learned_addr learned my addr 172.21.15.47:0/3641194258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.725+0000 7f1c96d9d700 1 -- 172.21.15.47:0/3641194258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 msgr2=0x55ee61e500a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:27:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.725+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/3641194258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 0x55ee61e500a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.725+0000 7f1c96d9d700 1 -- 172.21.15.47:0/3641194258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 msgr2=0x55ee61e48920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:27:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.725+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/3641194258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e48920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.725+0000 7f1c96d9d700 1 -- 172.21.15.47:0/3641194258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee61e631b0 con 0x55ee61e4dd50 2024-05-11T12:27:52.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.726+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/3641194258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e4dd50 0x55ee61e54ca0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1c900a6250 tx=0x7f1c900a61c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6114b9a1af00c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:52.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.728+0000 7f1c7ffff700 1 -- 172.21.15.47:0/3641194258 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c900aa1d0 con 0x55ee61e4dd50 2024-05-11T12:27:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.728+0000 7f1c7ffff700 1 -- 172.21.15.47:0/3641194258 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c900ae580 con 0x55ee61e4dd50 2024-05-11T12:27:52.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.729+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3641194258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e4dd50 msgr2=0x55ee61e54ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.729+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3641194258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e4dd50 0x55ee61e54ca0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1c900a6250 tx=0x7f1c900a61c0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.729+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3641194258 shutdown_connections 2024-05-11T12:27:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.729+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3641194258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e4dd50 0x55ee61e54ca0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.729+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3641194258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e48920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.729+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/3641194258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 0x55ee61e500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.729+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3641194258 >> 172.21.15.47:0/3641194258 conn(0x55ee61d75e60 msgr2=0x55ee61e4f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:52.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.734+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3641194258 shutdown_connections 2024-05-11T12:27:52.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.735+0000 7f1ca7450b80 1 -- 172.21.15.47:0/3641194258 wait complete. 2024-05-11T12:27:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.735+0000 7f1ca7450b80 1 -- start start 2024-05-11T12:27:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.735+0000 7f1ca7450b80 1 Processor -- start 2024-05-11T12:27:52.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.736+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e5b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.736+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 0x55ee61e5b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.736+0000 7f1ca7450b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e58390 0x55ee61e58810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.736+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98073120 con 0x55ee61e58390 2024-05-11T12:27:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.736+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c98074590 con 0x55ee61e49280 2024-05-11T12:27:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.736+0000 7f1ca7450b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c980737f0 con 0x55ee61e49ba0 2024-05-11T12:27:52.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.736+0000 7f1ca7450b80 10 client.0 _ll_register_callbacks cb 0x55ee61d33920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:27:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.737+0000 7f1c96d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e58390 0x55ee61e58810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.737+0000 7f1c96d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e58390 0x55ee61e58810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59658/0 (socket says 172.21.15.47:59658) 2024-05-11T12:27:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.737+0000 7f1c96d9d700 1 -- 172.21.15.47:0/2580325499 learned_addr learned my addr 172.21.15.47:0/2580325499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:27:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.738+0000 7f1c96d9d700 1 -- 172.21.15.47:0/2580325499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 msgr2=0x55ee61e5b900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:27:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.738+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 0x55ee61e5b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.738+0000 7f1c96d9d700 1 -- 172.21.15.47:0/2580325499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 msgr2=0x55ee61e5b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:27:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.738+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e5b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.738+0000 7f1c96d9d700 1 -- 172.21.15.47:0/2580325499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee61e40560 con 0x55ee61e58390 2024-05-11T12:27:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.739+0000 7f1c96d9d700 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e58390 0x55ee61e58810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1c900090a0 tx=0x7f1c900aa050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd43090b830555be server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:27:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.749+0000 7f1c7dffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:27:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.749+0000 7f1c7dffb700 1 -- 172.21.15.47:0/2580325499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c900b2fa0 con 0x55ee61e58390 2024-05-11T12:27:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.749+0000 7f1c7dffb700 1 -- 172.21.15.47:0/2580325499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1c900074e0 con 0x55ee61e58390 2024-05-11T12:27:52.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.750+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee61da32e0 con 0x55ee61e58390 2024-05-11T12:27:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.759+0000 7f1c7dffb700 1 -- 172.21.15.47:0/2580325499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1c9001b3a0 con 0x55ee61e58390 2024-05-11T12:27:52.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.763+0000 7f1ca7450b80 10 client.15444 fetch_fsmap learned FSMap version 20 2024-05-11T12:27:52.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.763+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ee61ece900 con 0x55ee61e58390 2024-05-11T12:27:52.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.768+0000 7f1c7dffb700 1 -- 172.21.15.47:0/2580325499 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ee61da32e0 con 0x55ee61e58390 2024-05-11T12:27:52.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.769+0000 7f1c7dffb700 1 -- 172.21.15.47:0/2580325499 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1c90011070 con 0x55ee61e58390 2024-05-11T12:27:52.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.769+0000 7f1ca7450b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:27:52.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.769+0000 7f1ca7450b80 10 client.15444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:27:52.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.769+0000 7f1ca7450b80 20 client.15444 populate_metadata read hostname 'smithi047' 2024-05-11T12:27:52.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.769+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61ecef30 0x55ee61ed1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:27:52.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.769+0000 7f1ca7450b80 4 client.15444 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:27:52.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.769+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee61c1b250 con 0x55ee61ecef30 2024-05-11T12:27:52.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.770+0000 7f1c9659c700 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61ecef30 0x55ee61ed1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:27:52.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.785+0000 7f1c9659c700 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61ecef30 0x55ee61ed1350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b52f28458154c13 server_cookie=c6bd6be760fbc21e in_seq=0 out_seq=0 2024-05-11T12:27:52.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.785+0000 7f1c7dffb700 10 client.15444 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.792+0000 7f1c7dffb700 1 -- 172.21.15.47:0/2580325499 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ee61c1b250 con 0x55ee61ecef30 2024-05-11T12:27:52.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.792+0000 7f1c7dffb700 10 client.15444 handle_command_reply: tid=0 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.792+0000 7f1ca7450b80 1 client.15444 shutdown 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e58390 msgr2=0x55ee61e58810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e58390 0x55ee61e58810 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f1c900090a0 tx=0x7f1c900aa050 comp rx=0 tx=0).stop 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 shutdown_connections 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ee61ecef30 0x55ee61ed1350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ee61e58390 0x55ee61e58810 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ee61e49280 0x55ee61e5b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 --2- 172.21.15.47:0/2580325499 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ee61e49ba0 0x55ee61e5b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 >> 172.21.15.47:0/2580325499 conn(0x55ee61d75e60 msgr2=0x55ee61e487a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.793+0000 7f1c7dffb700 0 client.15444 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:27:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.796+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 shutdown_connections 2024-05-11T12:27:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.797+0000 7f1ca7450b80 1 -- 172.21.15.47:0/2580325499 wait complete. 2024-05-11T12:27:52.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.797+0000 7f1ca7450b80 20 client.15444 trim_cache size 0 max 16384 2024-05-11T12:27:52.798 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:27:52.798 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:27:52.798 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:27:52.798 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:27:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c840b2a80 msgr2=0x7f1c840b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c840b2a80 0x7f1c840b4f40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1c80009d20 tx=0x7f1c80009260 comp rx=0 tx=0).stop 2024-05-11T12:27:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c98108420 msgr2=0x7f1c9819c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:27:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c98108420 0x7f1c9819c1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000b070 tx=0x7f1c90004f40 comp rx=0 tx=0).stop 2024-05-11T12:27:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 shutdown_connections 2024-05-11T12:27:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c98108420 0x7f1c9819c1c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c98102610 0x7f1c98197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c840b2a80 0x7f1c840b4f40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 --2- 172.21.15.47:0/2476782583 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c98102fb0 0x7f1c981981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:27:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 >> 172.21.15.47:0/2476782583 conn(0x7f1c98079290 msgr2=0x7f1c981004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:27:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 shutdown_connections 2024-05-11T12:27:52.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:27:52.799+0000 7f1c9759e700 1 -- 172.21.15.47:0/2476782583 wait complete. 2024-05-11T12:27:53.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:53 smithi047 ceph-mon[30375]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 856 MiB/s wr, 1.67k op/s 2024-05-11T12:27:53.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:53 smithi064 ceph-mon[34537]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 856 MiB/s wr, 1.67k op/s 2024-05-11T12:27:53.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:53 smithi050 ceph-mon[34072]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 856 MiB/s wr, 1.67k op/s 2024-05-11T12:27:55.213 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:27:55.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:27:55.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:55 smithi050 ceph-mon[34072]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 869 MiB/s wr, 1.67k op/s 2024-05-11T12:27:55.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:55 smithi047 ceph-mon[30375]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 869 MiB/s wr, 1.67k op/s 2024-05-11T12:27:55.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:55 smithi064 ceph-mon[34537]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 869 MiB/s wr, 1.67k op/s 2024-05-11T12:27:56.415 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:27:57.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:57 smithi050 ceph-mon[34072]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 741 MiB/s wr, 1.43k op/s 2024-05-11T12:27:57.473 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:57 smithi047 ceph-mon[30375]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 741 MiB/s wr, 1.43k op/s 2024-05-11T12:27:57.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:57 smithi064 ceph-mon[34537]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 741 MiB/s wr, 1.43k op/s 2024-05-11T12:27:59.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:27:59 smithi050 ceph-mon[34072]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.99k op/s 2024-05-11T12:27:59.539 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:27:59 smithi047 ceph-mon[30375]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.99k op/s 2024-05-11T12:27:59.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:27:59 smithi064 ceph-mon[34537]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 908 MiB/s wr, 1.99k op/s 2024-05-11T12:28:01.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:01 smithi050 ceph-mon[34072]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 768 MiB/s wr, 1.88k op/s 2024-05-11T12:28:01.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:01 smithi047 ceph-mon[30375]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 768 MiB/s wr, 1.88k op/s 2024-05-11T12:28:01.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:01 smithi064 ceph-mon[34537]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 768 MiB/s wr, 1.88k op/s 2024-05-11T12:28:01.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.901+0000 7f1c588dd700 1 -- 172.21.15.47:0/639771616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c540758f0 msgr2=0x7f1c54075cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:01.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.901+0000 7f1c588dd700 1 --2- 172.21.15.47:0/639771616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c540758f0 0x7f1c54075cd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009990 tx=0x7f1c48009ca0 comp rx=0 tx=0).stop 2024-05-11T12:28:01.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.901+0000 7f1c588dd700 1 -- 172.21.15.47:0/639771616 shutdown_connections 2024-05-11T12:28:01.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.901+0000 7f1c588dd700 1 --2- 172.21.15.47:0/639771616 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c54114490 0x7f1c54116880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:01.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.901+0000 7f1c588dd700 1 --2- 172.21.15.47:0/639771616 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c540758f0 0x7f1c54075cd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:01.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.901+0000 7f1c588dd700 1 --2- 172.21.15.47:0/639771616 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c54076210 0x7f1c54113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:01.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.901+0000 7f1c588dd700 1 -- 172.21.15.47:0/639771616 >> 172.21.15.47:0/639771616 conn(0x7f1c540fefd0 msgr2=0x7f1c541013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.911+0000 7f1c588dd700 1 -- 172.21.15.47:0/639771616 shutdown_connections 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.911+0000 7f1c588dd700 1 -- 172.21.15.47:0/639771616 wait complete. 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.912+0000 7f1c588dd700 1 Processor -- start 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.913+0000 7f1c588dd700 1 -- start start 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.913+0000 7f1c588dd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c540758f0 0x7f1c54113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.913+0000 7f1c588dd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c54114490 0x7f1c5410cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c588dd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c5410d1a0 0x7f1c5410d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c588dd700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54119da0 con 0x7f1c54114490 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c588dd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54119c20 con 0x7f1c5410d1a0 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c588dd700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1c54119f20 con 0x7f1c540758f0 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c540758f0 0x7f1c54113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c537fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c540758f0 0x7f1c54113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:45770/0 (socket says 172.21.15.47:45770) 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c537fe700 1 -- 172.21.15.47:0/1085091584 learned_addr learned my addr 172.21.15.47:0/1085091584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:01.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.914+0000 7f1c52ffd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c54114490 0x7f1c5410cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.915+0000 7f1c53fff700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c5410d1a0 0x7f1c5410d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.915+0000 7f1c537fe700 1 -- 172.21.15.47:0/1085091584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c5410d1a0 msgr2=0x7f1c5410d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.915+0000 7f1c537fe700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c5410d1a0 0x7f1c5410d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.915+0000 7f1c537fe700 1 -- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c54114490 msgr2=0x7f1c5410cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.915+0000 7f1c537fe700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c54114490 0x7f1c5410cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.915+0000 7f1c537fe700 1 -- 172.21.15.47:0/1085091584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c54111730 con 0x7f1c540758f0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.916+0000 7f1c537fe700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c540758f0 0x7f1c54113b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c480057c0 tx=0x7f1c480057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.916+0000 7f1c50ff9700 1 -- 172.21.15.47:0/1085091584 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48026070 con 0x7f1c540758f0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.917+0000 7f1c50ff9700 1 -- 172.21.15.47:0/1085091584 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c48021410 con 0x7f1c540758f0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.917+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c541119c0 con 0x7f1c540758f0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.917+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c54111ef0 con 0x7f1c540758f0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.917+0000 7f1c50ff9700 1 -- 172.21.15.47:0/1085091584 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c48017650 con 0x7f1c540758f0 2024-05-11T12:28:01.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.925+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c40005330 con 0x7f1c540758f0 2024-05-11T12:28:01.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.926+0000 7f1c50ff9700 1 -- 172.21.15.47:0/1085091584 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1c48009d10 con 0x7f1c540758f0 2024-05-11T12:28:01.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.928+0000 7f1c50ff9700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c3c0b29c0 0x7f1c3c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:01.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.929+0000 7f1c52ffd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c3c0b29c0 0x7f1c3c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:01.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.930+0000 7f1c52ffd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c3c0b29c0 0x7f1c3c0b4e80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1c5410e3d0 tx=0x7f1c44008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:01.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.930+0000 7f1c50ff9700 1 -- 172.21.15.47:0/1085091584 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1c480e2930 con 0x7f1c540758f0 2024-05-11T12:28:01.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:01.934+0000 7f1c50ff9700 1 -- 172.21.15.47:0/1085091584 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c48013070 con 0x7f1c540758f0 2024-05-11T12:28:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.595+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c40005620 con 0x7f1c540758f0 2024-05-11T12:28:02.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.598+0000 7f1c50ff9700 1 -- 172.21.15.47:0/1085091584 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f1c480aa5b0 con 0x7f1c540758f0 2024-05-11T12:28:02.601 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:28:02.601 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:28:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.608+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c3c0b29c0 msgr2=0x7f1c3c0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.608+0000 7f1c588dd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c3c0b29c0 0x7f1c3c0b4e80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1c5410e3d0 tx=0x7f1c44008040 comp rx=0 tx=0).stop 2024-05-11T12:28:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.608+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c540758f0 msgr2=0x7f1c54113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:02.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.608+0000 7f1c588dd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c540758f0 0x7f1c54113b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1c480057c0 tx=0x7f1c480057f0 comp rx=0 tx=0).stop 2024-05-11T12:28:02.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.612+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 shutdown_connections 2024-05-11T12:28:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.612+0000 7f1c588dd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1c3c0b29c0 0x7f1c3c0b4e80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.612+0000 7f1c588dd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1c5410d1a0 0x7f1c5410d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.612+0000 7f1c588dd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1c540758f0 0x7f1c54113b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.612+0000 7f1c588dd700 1 --2- 172.21.15.47:0/1085091584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c54114490 0x7f1c5410cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:02.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.612+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 >> 172.21.15.47:0/1085091584 conn(0x7f1c540fefd0 msgr2=0x7f1c541009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:02.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.615+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 shutdown_connections 2024-05-11T12:28:02.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:02.615+0000 7f1c588dd700 1 -- 172.21.15.47:0/1085091584 wait complete. 2024-05-11T12:28:02.625 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:28:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:03 smithi050 ceph-mon[34072]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 2.14k op/s 2024-05-11T12:28:03.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:03 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1085091584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:28:03.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:03 smithi047 ceph-mon[30375]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 2.14k op/s 2024-05-11T12:28:03.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:03 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1085091584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:28:03.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:03 smithi064 ceph-mon[34537]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 849 MiB/s wr, 2.14k op/s 2024-05-11T12:28:03.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:03 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1085091584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:28:04.698 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:28:05.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:05 smithi050 ceph-mon[34072]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 866 MiB/s wr, 2.15k op/s 2024-05-11T12:28:05.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:05 smithi047 ceph-mon[30375]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 866 MiB/s wr, 2.15k op/s 2024-05-11T12:28:05.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:05 smithi064 ceph-mon[34537]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 866 MiB/s wr, 2.15k op/s 2024-05-11T12:28:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:28:07.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:07 smithi047 ceph-mon[30375]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 732 MiB/s wr, 1.92k op/s 2024-05-11T12:28:07.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:07 smithi064 ceph-mon[34537]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 732 MiB/s wr, 1.92k op/s 2024-05-11T12:28:07.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:07 smithi050 ceph-mon[34072]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 732 MiB/s wr, 1.92k op/s 2024-05-11T12:28:09.479 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:09 smithi047 ceph-mon[30375]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 901 MiB/s wr, 2.22k op/s 2024-05-11T12:28:09.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:09 smithi064 ceph-mon[34537]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 901 MiB/s wr, 2.22k op/s 2024-05-11T12:28:09.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:09 smithi050 ceph-mon[34072]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 901 MiB/s wr, 2.22k op/s 2024-05-11T12:28:10.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.575+0000 7fa06af1c700 1 -- 172.21.15.47:0/1756464140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064102d50 msgr2=0x7fa064107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.575+0000 7fa06af1c700 1 --2- 172.21.15.47:0/1756464140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064102d50 0x7fa064107220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa0540098f0 tx=0x7fa054009c00 comp rx=0 tx=0).stop 2024-05-11T12:28:10.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.580+0000 7fa06af1c700 1 -- 172.21.15.47:0/1756464140 shutdown_connections 2024-05-11T12:28:10.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.580+0000 7fa06af1c700 1 --2- 172.21.15.47:0/1756464140 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064107840 0x7fa064107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.580+0000 7fa06af1c700 1 --2- 172.21.15.47:0/1756464140 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa064102430 0x7fa064102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.580+0000 7fa06af1c700 1 --2- 172.21.15.47:0/1756464140 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064102d50 0x7fa064107220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.580+0000 7fa06af1c700 1 -- 172.21.15.47:0/1756464140 >> 172.21.15.47:0/1756464140 conn(0x7fa0640fde70 msgr2=0x7fa064100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:10.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.589+0000 7fa06af1c700 1 -- 172.21.15.47:0/1756464140 shutdown_connections 2024-05-11T12:28:10.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.589+0000 7fa06af1c700 1 -- 172.21.15.47:0/1756464140 wait complete. 2024-05-11T12:28:10.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.590+0000 7fa06af1c700 1 Processor -- start 2024-05-11T12:28:10.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.592+0000 7fa06af1c700 1 -- start start 2024-05-11T12:28:10.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.592+0000 7fa06af1c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064102430 0x7fa0641a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.592+0000 7fa06af1c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa064102d50 0x7fa0641a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.592+0000 7fa06af1c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064107840 0x7fa0641a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.592+0000 7fa06af1c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa064119d60 con 0x7fa064102430 2024-05-11T12:28:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.592+0000 7fa06af1c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa064119be0 con 0x7fa064107840 2024-05-11T12:28:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.592+0000 7fa06af1c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa064119ee0 con 0x7fa064102d50 2024-05-11T12:28:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.593+0000 7fa069f1a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064102430 0x7fa0641a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.593+0000 7fa06a71b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064107840 0x7fa0641a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.593+0000 7fa06a71b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064107840 0x7fa0641a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:56644/0 (socket says 172.21.15.47:56644) 2024-05-11T12:28:10.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.593+0000 7fa06a71b700 1 -- 172.21.15.47:0/2758954313 learned_addr learned my addr 172.21.15.47:0/2758954313 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.594+0000 7fa069719700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa064102d50 0x7fa0641a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.598+0000 7fa06a71b700 1 -- 172.21.15.47:0/2758954313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa064102d50 msgr2=0x7fa0641a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.598+0000 7fa06a71b700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa064102d50 0x7fa0641a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.598+0000 7fa06a71b700 1 -- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064102430 msgr2=0x7fa0641a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.598+0000 7fa06a71b700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064102430 0x7fa0641a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.598+0000 7fa06a71b700 1 -- 172.21.15.47:0/2758954313 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0641a5230 con 0x7fa064107840 2024-05-11T12:28:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.599+0000 7fa06a71b700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064107840 0x7fa0641a4af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00b970 tx=0x7fa05c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.601+0000 7fa05affd700 1 -- 172.21.15.47:0/2758954313 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c019070 con 0x7fa064107840 2024-05-11T12:28:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.601+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0641a5520 con 0x7fa064107840 2024-05-11T12:28:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.601+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0641087f0 con 0x7fa064107840 2024-05-11T12:28:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.602+0000 7fa05affd700 1 -- 172.21.15.47:0/2758954313 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c00cda0 con 0x7fa064107840 2024-05-11T12:28:10.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.602+0000 7fa05affd700 1 -- 172.21.15.47:0/2758954313 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c007370 con 0x7fa064107840 2024-05-11T12:28:10.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.618+0000 7fa076a14b80 1 Processor -- start 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.626+0000 7fa076a14b80 1 -- start start 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.627+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4000 0x5611a29a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.627+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4920 0x5611a299e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.627+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a299ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.627+0000 7fa076a14b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5611a28f0940 con 0x5611a29a4920 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.627+0000 7fa076a14b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5611a2855da0 con 0x5611a299eb30 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.627+0000 7fa076a14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5611a28ec900 con 0x5611a29a4000 2024-05-11T12:28:10.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.627+0000 7fa05affd700 1 -- 172.21.15.47:0/2758954313 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa05c01c070 con 0x7fa064107840 2024-05-11T12:28:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.629+0000 7fa069719700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4000 0x5611a29a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.629+0000 7fa069719700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4000 0x5611a29a43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:50164/0 (socket says 172.21.15.47:50164) 2024-05-11T12:28:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.629+0000 7fa069719700 1 -- 172.21.15.47:0/465414245 learned_addr learned my addr 172.21.15.47:0/465414245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.629+0000 7fa069719700 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a299ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.630+0000 7fa069f1a700 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4920 0x5611a299e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.630+0000 7fa069719700 1 -- 172.21.15.47:0/465414245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 msgr2=0x5611a299ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.630+0000 7fa069719700 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a299ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.630+0000 7fa069719700 1 -- 172.21.15.47:0/465414245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4920 msgr2=0x5611a299e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.630+0000 7fa069719700 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4920 0x5611a299e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.630+0000 7fa069719700 1 -- 172.21.15.47:0/465414245 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a283d8d0 con 0x5611a29a4000 2024-05-11T12:28:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.631+0000 7fa05affd700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0500b2a90 0x7fa0500b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.632+0000 7fa05affd700 1 -- 172.21.15.47:0/2758954313 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fa05c0e16b0 con 0x7fa064107840 2024-05-11T12:28:10.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.633+0000 7fa069719700 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4000 0x5611a29a43e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa0641031b0 tx=0x7fa054009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7f67b34ce4e8636 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:10.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.635+0000 7fa069f1a700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0500b2a90 0x7fa0500b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.636+0000 7fa04ffff700 1 -- 172.21.15.47:0/465414245 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054020040 con 0x5611a29a4000 2024-05-11T12:28:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.636+0000 7fa04ffff700 1 -- 172.21.15.47:0/465414245 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa054005b00 con 0x5611a29a4000 2024-05-11T12:28:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.636+0000 7fa04ffff700 1 -- 172.21.15.47:0/465414245 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05400b560 con 0x5611a29a4000 2024-05-11T12:28:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.638+0000 7fa076a14b80 1 -- 172.21.15.47:0/465414245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4000 msgr2=0x5611a29a43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.638+0000 7fa076a14b80 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4000 0x5611a29a43e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa0641031b0 tx=0x7fa054009520 comp rx=0 tx=0).stop 2024-05-11T12:28:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.646+0000 7fa069f1a700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0500b2a90 0x7fa0500b4f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa060005fd0 tx=0x7fa060009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.646+0000 7fa076a14b80 1 -- 172.21.15.47:0/465414245 shutdown_connections 2024-05-11T12:28:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.647+0000 7fa076a14b80 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a299ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.647+0000 7fa076a14b80 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4000 0x5611a29a43e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.647+0000 7fa076a14b80 1 --2- 172.21.15.47:0/465414245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4920 0x5611a299e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.647+0000 7fa076a14b80 1 -- 172.21.15.47:0/465414245 >> 172.21.15.47:0/465414245 conn(0x5611a28cab70 msgr2=0x5611a2999090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.653+0000 7fa076a14b80 1 -- 172.21.15.47:0/465414245 shutdown_connections 2024-05-11T12:28:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.653+0000 7fa076a14b80 1 -- 172.21.15.47:0/465414245 wait complete. 2024-05-11T12:28:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.654+0000 7fa076a14b80 1 -- start start 2024-05-11T12:28:10.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.655+0000 7fa076a14b80 1 Processor -- start 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.656+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a29aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.656+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.657+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29a7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.657+0000 7fa076a14b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5611a29ab3e0 con 0x5611a29a4000 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.657+0000 7fa076a14b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5611a28a9ed0 con 0x5611a299eb30 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.657+0000 7fa076a14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5611a29ab560 con 0x5611a29a4920 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.657+0000 7fa076a14b80 10 client.0 _ll_register_callbacks cb 0x5611a2888950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.658+0000 7fa069f1a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a29aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.658+0000 7fa069719700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.658+0000 7fa06a71b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29a7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.658+0000 7fa069719700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29a79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36754/0 (socket says 172.21.15.47:36754) 2024-05-11T12:28:10.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.658+0000 7fa069719700 1 -- 172.21.15.47:0/29530480 learned_addr learned my addr 172.21.15.47:0/29530480 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:10.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.659+0000 7fa069719700 1 -- 172.21.15.47:0/29530480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 msgr2=0x5611a29a7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.659+0000 7fa069719700 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29a7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.659+0000 7fa069719700 1 -- 172.21.15.47:0/29530480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 msgr2=0x5611a29aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.659+0000 7fa069719700 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a29aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.659+0000 7fa069719700 1 -- 172.21.15.47:0/29530480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a28a7760 con 0x5611a29a4000 2024-05-11T12:28:10.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.659+0000 7fa069719700 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29a79c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa054005950 tx=0x7fa054005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a86d4b109d05a9d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:10.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.659+0000 7fa04d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:28:10.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.660+0000 7fa04d7fa700 1 -- 172.21.15.47:0/29530480 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054003680 con 0x5611a29a4000 2024-05-11T12:28:10.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.660+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa054005b00 con 0x5611a29a4000 2024-05-11T12:28:10.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.660+0000 7fa04d7fa700 1 -- 172.21.15.47:0/29530480 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fa054003820 con 0x5611a29a4000 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa04d7fa700 1 -- 172.21.15.47:0/29530480 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa054003a40 con 0x5611a29a4000 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 10 client.15459 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5611a29ab3e0 con 0x5611a29a4000 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa04d7fa700 1 -- 172.21.15.47:0/29530480 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa054003be0 con 0x5611a29a4000 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa04d7fa700 1 -- 172.21.15.47:0/29530480 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa05403b480 con 0x5611a29a4000 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 20 client.15459 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a29b7de0 0x5611a29ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 4 client.15459 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.661+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa054005b00 con 0x5611a29b7de0 2024-05-11T12:28:10.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.662+0000 7fa069f1a700 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a29b7de0 0x5611a29ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.665+0000 7fa069f1a700 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a29b7de0 0x5611a29ba200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8613c248de5e0f6 server_cookie=76f24b2a5e823c0a in_seq=0 out_seq=0 2024-05-11T12:28:10.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.665+0000 7fa04d7fa700 10 client.15459 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.668+0000 7fa04d7fa700 1 -- 172.21.15.47:0/29530480 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fa054005b00 con 0x5611a29b7de0 2024-05-11T12:28:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.668+0000 7fa04d7fa700 10 client.15459 handle_command_reply: tid=0 2024-05-11T12:28:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.668+0000 7fa076a14b80 1 client.15459 shutdown 2024-05-11T12:28:10.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.669+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 msgr2=0x5611a29a79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.669+0000 7fa076a14b80 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29a79c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa054005950 tx=0x7fa054005270 comp rx=0 tx=0).stop 2024-05-11T12:28:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.670+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 shutdown_connections 2024-05-11T12:28:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.670+0000 7fa076a14b80 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a29b7de0 0x5611a29ba200 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.670+0000 7fa076a14b80 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29a7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.670+0000 7fa076a14b80 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a299eb30 0x5611a29aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.670+0000 7fa076a14b80 1 --2- 172.21.15.47:0/29530480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29a79c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.670+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 >> 172.21.15.47:0/29530480 conn(0x5611a28cab70 msgr2=0x5611a2998660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:10.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.670+0000 7fa04d7fa700 0 client.15459 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.672+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 shutdown_connections 2024-05-11T12:28:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.672+0000 7fa076a14b80 1 -- 172.21.15.47:0/29530480 wait complete. 2024-05-11T12:28:10.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.673+0000 7fa076a14b80 20 client.15459 trim_cache size 0 max 16384 2024-05-11T12:28:10.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.709+0000 7fa076a14b80 1 Processor -- start 2024-05-11T12:28:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.713+0000 7fa076a14b80 1 -- start start 2024-05-11T12:28:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.713+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a299eb30 0x5611a299e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.713+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4000 0x5611a29a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.713+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.713+0000 7fa076a14b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa04800b760 con 0x5611a299eb30 2024-05-11T12:28:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.713+0000 7fa076a14b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa04800dfe0 con 0x5611a29a4000 2024-05-11T12:28:10.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.713+0000 7fa076a14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa04800bdd0 con 0x5611a29a4920 2024-05-11T12:28:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.716+0000 7fa069719700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4000 0x5611a29a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.716+0000 7fa06a71b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.716+0000 7fa069f1a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a299eb30 0x5611a299e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.716+0000 7fa069f1a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a299eb30 0x5611a299e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36766/0 (socket says 172.21.15.47:36766) 2024-05-11T12:28:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.716+0000 7fa069f1a700 1 -- 172.21.15.47:0/165243236 learned_addr learned my addr 172.21.15.47:0/165243236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.718+0000 7fa06a71b700 1 -- 172.21.15.47:0/165243236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4000 msgr2=0x5611a29a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.718+0000 7fa06a71b700 1 --2- 172.21.15.47:0/165243236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4000 0x5611a29a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.718+0000 7fa06a71b700 1 -- 172.21.15.47:0/165243236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a299eb30 msgr2=0x5611a299e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.718+0000 7fa06a71b700 1 --2- 172.21.15.47:0/165243236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a299eb30 0x5611a299e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.718+0000 7fa06a71b700 1 -- 172.21.15.47:0/165243236 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a299ae70 con 0x5611a29a4920 2024-05-11T12:28:10.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.720+0000 7fa06a71b700 1 --2- 172.21.15.47:0/165243236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29b4500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0a88a0 tx=0x7fa05c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fc7856ce91c0e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:10.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.720+0000 7fa04f7fe700 1 -- 172.21.15.47:0/165243236 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0a86b0 con 0x5611a29a4920 2024-05-11T12:28:10.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.720+0000 7fa04f7fe700 1 -- 172.21.15.47:0/165243236 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c0c9e50 con 0x5611a29a4920 2024-05-11T12:28:10.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.720+0000 7fa04f7fe700 1 -- 172.21.15.47:0/165243236 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0b1a00 con 0x5611a29a4920 2024-05-11T12:28:10.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.722+0000 7fa076a14b80 1 -- 172.21.15.47:0/165243236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 msgr2=0x5611a29b4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.722+0000 7fa076a14b80 1 --2- 172.21.15.47:0/165243236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29b4500 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa05c0a88a0 tx=0x7fa05c004ad0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.726+0000 7fa076a14b80 1 -- 172.21.15.47:0/165243236 shutdown_connections 2024-05-11T12:28:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.726+0000 7fa076a14b80 1 --2- 172.21.15.47:0/165243236 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a29a4920 0x5611a29b4500 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.726+0000 7fa076a14b80 1 --2- 172.21.15.47:0/165243236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a299eb30 0x5611a299e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.726+0000 7fa076a14b80 1 --2- 172.21.15.47:0/165243236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4000 0x5611a29a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.726+0000 7fa076a14b80 1 -- 172.21.15.47:0/165243236 >> 172.21.15.47:0/165243236 conn(0x5611a28cab70 msgr2=0x5611a29a1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:10.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.729+0000 7fa076a14b80 1 -- 172.21.15.47:0/165243236 shutdown_connections 2024-05-11T12:28:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.731+0000 7fa076a14b80 1 -- 172.21.15.47:0/165243236 wait complete. 2024-05-11T12:28:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.732+0000 7fa076a14b80 1 -- start start 2024-05-11T12:28:10.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.732+0000 7fa076a14b80 1 Processor -- start 2024-05-11T12:28:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.733+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a299eb30 0x5611a29b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.733+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.733+0000 7fa076a14b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4920 0x5611a29bf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.733+0000 7fa076a14b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa04800c4a0 con 0x5611a29a4000 2024-05-11T12:28:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.734+0000 7fa076a14b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa04800d910 con 0x5611a29a4920 2024-05-11T12:28:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.734+0000 7fa076a14b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa04800cb70 con 0x5611a299eb30 2024-05-11T12:28:10.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.734+0000 7fa076a14b80 10 client.0 _ll_register_callbacks cb 0x5611a2888950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.735+0000 7fa06a71b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4920 0x5611a29bf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.736+0000 7fa069f1a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a299eb30 0x5611a29b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.736+0000 7fa06a71b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4920 0x5611a29bf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:56672/0 (socket says 172.21.15.47:56672) 2024-05-11T12:28:10.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.736+0000 7fa06a71b700 1 -- 172.21.15.47:0/2555384069 learned_addr learned my addr 172.21.15.47:0/2555384069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.737+0000 7fa069719700 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.740+0000 7fa069719700 1 -- 172.21.15.47:0/2555384069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a299eb30 msgr2=0x5611a29b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.740+0000 7fa069719700 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a299eb30 0x5611a29b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.740+0000 7fa069719700 1 -- 172.21.15.47:0/2555384069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4920 msgr2=0x5611a29bf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.740+0000 7fa069719700 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4920 0x5611a29bf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.740+0000 7fa069719700 1 -- 172.21.15.47:0/2555384069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a29bff50 con 0x5611a29a4000 2024-05-11T12:28:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.740+0000 7fa069719700 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29bf140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0540092a0 tx=0x7fa054003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ec607dadb6e17f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:10.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.740+0000 7fa04d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:28:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.742+0000 7fa04d7fa700 1 -- 172.21.15.47:0/2555384069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa054005b00 con 0x5611a29a4000 2024-05-11T12:28:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.742+0000 7fa04d7fa700 1 -- 172.21.15.47:0/2555384069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fa05403b690 con 0x5611a29a4000 2024-05-11T12:28:10.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.746+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa05c0c9e50 con 0x5611a29a4000 2024-05-11T12:28:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.748+0000 7fa04d7fa700 1 -- 172.21.15.47:0/2555384069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa05400b560 con 0x5611a29a4000 2024-05-11T12:28:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.749+0000 7fa076a14b80 10 client.15465 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.749+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5611a2a22780 con 0x5611a29a4000 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.753+0000 7fa04d7fa700 1 -- 172.21.15.47:0/2555384069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa05c0c9e50 con 0x5611a29a4000 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.753+0000 7fa04d7fa700 1 -- 172.21.15.47:0/2555384069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fa054027470 con 0x5611a29a4000 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.754+0000 7fa076a14b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.754+0000 7fa076a14b80 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.754+0000 7fa076a14b80 20 client.15465 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.755+0000 7fa076a14b80 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a2a22db0 0x5611a2a251d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.755+0000 7fa076a14b80 4 client.15465 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:28:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.755+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611a28fe320 con 0x5611a2a22db0 2024-05-11T12:28:10.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.756+0000 7fa069f1a700 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a2a22db0 0x5611a2a251d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:10.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.761+0000 7fa069f1a700 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a2a22db0 0x5611a2a251d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc8b8f1faa9831f7 server_cookie=54ef5c8b898d808b in_seq=0 out_seq=0 2024-05-11T12:28:10.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.761+0000 7fa04d7fa700 10 client.15465 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:10.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.766+0000 7fa04d7fa700 1 -- 172.21.15.47:0/2555384069 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611a28fe320 con 0x5611a2a22db0 2024-05-11T12:28:10.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.766+0000 7fa04d7fa700 10 client.15465 handle_command_reply: tid=0 2024-05-11T12:28:10.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.767+0000 7fa076a14b80 1 client.15465 shutdown 2024-05-11T12:28:10.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.771+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 msgr2=0x5611a29bf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.771+0000 7fa076a14b80 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29bf140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa0540092a0 tx=0x7fa054003e90 comp rx=0 tx=0).stop 2024-05-11T12:28:10.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 shutdown_connections 2024-05-11T12:28:10.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa076a14b80 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5611a2a22db0 0x5611a2a251d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa076a14b80 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5611a29a4920 0x5611a29bf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa076a14b80 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5611a299eb30 0x5611a29b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa076a14b80 1 --2- 172.21.15.47:0/2555384069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5611a29a4000 0x5611a29bf140 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 >> 172.21.15.47:0/2555384069 conn(0x5611a28cab70 msgr2=0x5611a29a4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:10.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa04d7fa700 0 client.15465 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:10.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.772+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 shutdown_connections 2024-05-11T12:28:10.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.773+0000 7fa076a14b80 1 -- 172.21.15.47:0/2555384069 wait complete. 2024-05-11T12:28:10.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.774+0000 7fa076a14b80 20 client.15465 trim_cache size 0 max 16384 2024-05-11T12:28:10.783 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0500b2a90 msgr2=0x7fa0500b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0500b2a90 0x7fa0500b4f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa060005fd0 tx=0x7fa060009310 comp rx=0 tx=0).stop 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064107840 msgr2=0x7fa0641a4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064107840 0x7fa0641a4af0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00b970 tx=0x7fa05c00bd30 comp rx=0 tx=0).stop 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 shutdown_connections 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0500b2a90 0x7fa0500b4f50 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa060005fd0 tx=0x7fa060009310 comp rx=0 tx=0).stop 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa064107840 0x7fa0641a4af0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa064102430 0x7fa0641a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 --2- 172.21.15.47:0/2758954313 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa064102d50 0x7fa0641a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:10.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.783+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 >> 172.21.15.47:0/2758954313 conn(0x7fa0640fde70 msgr2=0x7fa0641051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.784+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 shutdown_connections 2024-05-11T12:28:10.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:10.784+0000 7fa06af1c700 1 -- 172.21.15.47:0/2758954313 wait complete. 2024-05-11T12:28:11.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:11 smithi047 ceph-mon[30375]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 765 MiB/s wr, 1.70k op/s 2024-05-11T12:28:11.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:11 smithi064 ceph-mon[34537]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 765 MiB/s wr, 1.70k op/s 2024-05-11T12:28:11.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:11 smithi050 ceph-mon[34072]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 765 MiB/s wr, 1.70k op/s 2024-05-11T12:28:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:13 smithi047 ceph-mon[30375]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 853 MiB/s wr, 1.75k op/s 2024-05-11T12:28:13.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:13 smithi064 ceph-mon[34537]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 853 MiB/s wr, 1.75k op/s 2024-05-11T12:28:13.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:13 smithi050 ceph-mon[34072]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 853 MiB/s wr, 1.75k op/s 2024-05-11T12:28:14.415 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:28:14.416 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:28:15.255 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:28:15.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:15 smithi047 ceph-mon[30375]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 870 MiB/s wr, 1.87k op/s 2024-05-11T12:28:15.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:15 smithi064 ceph-mon[34537]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 870 MiB/s wr, 1.87k op/s 2024-05-11T12:28:15.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:15 smithi050 ceph-mon[34072]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 870 MiB/s wr, 1.87k op/s 2024-05-11T12:28:17.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:17 smithi047 ceph-mon[30375]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 734 MiB/s wr, 1.77k op/s 2024-05-11T12:28:17.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:17 smithi064 ceph-mon[34537]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 734 MiB/s wr, 1.77k op/s 2024-05-11T12:28:17.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:17 smithi050 ceph-mon[34072]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 734 MiB/s wr, 1.77k op/s 2024-05-11T12:28:19.324 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:19 smithi047 ceph-mon[30375]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 889 MiB/s wr, 2.37k op/s 2024-05-11T12:28:19.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:19 smithi064 ceph-mon[34537]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 889 MiB/s wr, 2.37k op/s 2024-05-11T12:28:19.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:19 smithi050 ceph-mon[34072]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 889 MiB/s wr, 2.37k op/s 2024-05-11T12:28:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.158+0000 7f2a0359e700 1 -- 172.21.15.47:0/3403287523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a04109550 msgr2=0x7f2a04109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.158+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3403287523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a04109550 0x7f2a04109930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f29f40098f0 tx=0x7f29f4009c00 comp rx=0 tx=0).stop 2024-05-11T12:28:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.159+0000 7f2a0359e700 1 -- 172.21.15.47:0/3403287523 shutdown_connections 2024-05-11T12:28:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.159+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3403287523 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04103e60 0x7f2a04110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.159+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3403287523 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a04109550 0x7f2a04109930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.159+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3403287523 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a041034a0 0x7f2a04103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.159+0000 7f2a0359e700 1 -- 172.21.15.47:0/3403287523 >> 172.21.15.47:0/3403287523 conn(0x7f2a040fefd0 msgr2=0x7f2a041013f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.161+0000 7f2a0359e700 1 -- 172.21.15.47:0/3403287523 shutdown_connections 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.162+0000 7f2a0359e700 1 -- 172.21.15.47:0/3403287523 wait complete. 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.164+0000 7f2a0359e700 1 Processor -- start 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.164+0000 7f2a0359e700 1 -- start start 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a0359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a041034a0 0x7f2a0410f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a0359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a04103e60 0x7f2a0410f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a0359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04109550 0x7f2a0419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a0359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2a04113750 con 0x7f2a04109550 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a0359e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041135d0 con 0x7f2a041034a0 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a0359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2a041138d0 con 0x7f2a04103e60 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a02d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04109550 0x7f2a0419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a02d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04109550 0x7f2a0419c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51212/0 (socket says 172.21.15.47:51212) 2024-05-11T12:28:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.165+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3772067390 learned_addr learned my addr 172.21.15.47:0/3772067390 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.168+0000 7f2a0259c700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a041034a0 0x7f2a0410f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.169+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3772067390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a04103e60 msgr2=0x7f2a0410f980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:28:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.169+0000 7f2a02d9d700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a04103e60 0x7f2a0410f980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.169+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3772067390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a041034a0 msgr2=0x7f2a0410f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.169+0000 7f2a02d9d700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a041034a0 0x7f2a0410f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.169+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3772067390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0419c780 con 0x7f2a04109550 2024-05-11T12:28:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.173+0000 7f2a02d9d700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04109550 0x7f2a0419c240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007fd0 tx=0x7f29fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.177+0000 7f29f37fe700 1 -- 172.21.15.47:0/3772067390 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc00dbd0 con 0x7f2a04109550 2024-05-11T12:28:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.177+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0419ca70 con 0x7f2a04109550 2024-05-11T12:28:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.177+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0419d030 con 0x7f2a04109550 2024-05-11T12:28:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.178+0000 7f29f37fe700 1 -- 172.21.15.47:0/3772067390 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29fc00dd70 con 0x7f2a04109550 2024-05-11T12:28:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.178+0000 7f29f37fe700 1 -- 172.21.15.47:0/3772067390 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0133f0 con 0x7f2a04109550 2024-05-11T12:28:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.185+0000 7f2a13658b80 1 Processor -- start 2024-05-11T12:28:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.186+0000 7f2a13658b80 1 -- start start 2024-05-11T12:28:20.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.186+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0e9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.186+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.186+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0e4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.186+0000 7f2a13658b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d74e0359c0 con 0x55d74e0e3bb0 2024-05-11T12:28:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.186+0000 7f2a13658b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d74df9ae20 con 0x55d74e0e99a0 2024-05-11T12:28:20.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.186+0000 7f2a13658b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d74e031980 con 0x55d74e0e9080 2024-05-11T12:28:20.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.189+0000 7f2a0259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.189+0000 7f2a01d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.189+0000 7f2a0259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0e3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:35168/0 (socket says 172.21.15.47:35168) 2024-05-11T12:28:20.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.189+0000 7f2a0259c700 1 -- 172.21.15.47:0/4139939201 learned_addr learned my addr 172.21.15.47:0/4139939201 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:20.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.192+0000 7f29f37fe700 1 -- 172.21.15.47:0/3772067390 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f29fc01a020 con 0x7f2a04109550 2024-05-11T12:28:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.193+0000 7f2a0259c700 1 -- 172.21.15.47:0/4139939201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 msgr2=0x55d74e0e9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.193+0000 7f2a0259c700 1 --2- 172.21.15.47:0/4139939201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0e9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.193+0000 7f2a0259c700 1 -- 172.21.15.47:0/4139939201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 msgr2=0x55d74e0e4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.193+0000 7f2a0259c700 1 --2- 172.21.15.47:0/4139939201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0e4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.193+0000 7f2a0259c700 1 -- 172.21.15.47:0/4139939201 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74df82950 con 0x55d74e0e99a0 2024-05-11T12:28:20.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.198+0000 7f2a0259c700 1 --2- 172.21.15.47:0/4139939201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0e3590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000c00 tx=0x7f29f4009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b63780f11322825 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:20.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.199+0000 7f29f37fe700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29ec0b2a10 0x7f29ec0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.200+0000 7f29f0ff9700 1 -- 172.21.15.47:0/4139939201 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4024020 con 0x55d74e0e99a0 2024-05-11T12:28:20.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.200+0000 7f29f0ff9700 1 -- 172.21.15.47:0/4139939201 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f40054d0 con 0x55d74e0e99a0 2024-05-11T12:28:20.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.201+0000 7f2a13658b80 1 -- 172.21.15.47:0/4139939201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 msgr2=0x55d74e0e3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.202+0000 7f2a13658b80 1 --2- 172.21.15.47:0/4139939201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0e3590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f29f4000c00 tx=0x7f29f4009590 comp rx=0 tx=0).stop 2024-05-11T12:28:20.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.204+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29ec0b2a10 0x7f29ec0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.205+0000 7f29f37fe700 1 -- 172.21.15.47:0/3772067390 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f29fc0a5400 con 0x7f2a04109550 2024-05-11T12:28:20.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.205+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29ec0b2a10 0x7f29ec0b4ed0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a04077230 tx=0x7f29f800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.206+0000 7f2a13658b80 1 -- 172.21.15.47:0/4139939201 shutdown_connections 2024-05-11T12:28:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.206+0000 7f2a13658b80 1 --2- 172.21.15.47:0/4139939201 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0e4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.206+0000 7f2a13658b80 1 --2- 172.21.15.47:0/4139939201 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0e9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.206+0000 7f2a13658b80 1 --2- 172.21.15.47:0/4139939201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0e3590 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.206+0000 7f2a13658b80 1 -- 172.21.15.47:0/4139939201 >> 172.21.15.47:0/4139939201 conn(0x55d74e00fbf0 msgr2=0x55d74e0de110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:20.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.213+0000 7f2a13658b80 1 -- 172.21.15.47:0/4139939201 shutdown_connections 2024-05-11T12:28:20.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.214+0000 7f2a13658b80 1 -- 172.21.15.47:0/4139939201 wait complete. 2024-05-11T12:28:20.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.214+0000 7f2a13658b80 1 -- start start 2024-05-11T12:28:20.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 1 Processor -- start 2024-05-11T12:28:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d74e101470 con 0x55d74e0e3bb0 2024-05-11T12:28:20.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d74dfeef50 con 0x55d74e0e99a0 2024-05-11T12:28:20.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d74e1015f0 con 0x55d74e0e9080 2024-05-11T12:28:20.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.215+0000 7f2a13658b80 10 client.0 _ll_register_callbacks cb 0x55d74dfcd9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:20.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.217+0000 7f2a0259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.217+0000 7f2a0259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0ef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51224/0 (socket says 172.21.15.47:51224) 2024-05-11T12:28:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.218+0000 7f2a0259c700 1 -- 172.21.15.47:0/948634781 learned_addr learned my addr 172.21.15.47:0/948634781 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.218+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.218+0000 7f2a02d9d700 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.219+0000 7f2a0259c700 1 -- 172.21.15.47:0/948634781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 msgr2=0x55d74e0ec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.219+0000 7f2a0259c700 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0ec950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.219+0000 7f2a0259c700 1 -- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 msgr2=0x55d74e0ece90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.219+0000 7f2a0259c700 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0ece90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.219+0000 7f2a0259c700 1 -- 172.21.15.47:0/948634781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74dfec7e0 con 0x55d74e0e3bb0 2024-05-11T12:28:20.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.221+0000 7f2a0259c700 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0ef5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f29f4004970 tx=0x7f29f4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bc0436aa84e766 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.221+0000 7f29e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:28:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.223+0000 7f29e9ffb700 1 -- 172.21.15.47:0/948634781 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f402e060 con 0x55d74e0e3bb0 2024-05-11T12:28:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.223+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29f40054d0 con 0x55d74e0e3bb0 2024-05-11T12:28:20.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.223+0000 7f29e9ffb700 1 -- 172.21.15.47:0/948634781 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f29f4004f40 con 0x55d74e0e3bb0 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.224+0000 7f29e9ffb700 1 -- 172.21.15.47:0/948634781 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f29f4024040 con 0x55d74e0e3bb0 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.224+0000 7f2a13658b80 10 client.15474 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.224+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d74e10bc70 con 0x55d74e0e3bb0 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f29e9ffb700 1 -- 172.21.15.47:0/948634781 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f40054d0 con 0x55d74e0e3bb0 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f29e9ffb700 1 -- 172.21.15.47:0/948634781 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f29f401f070 con 0x55d74e0e3bb0 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f2a13658b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f2a13658b80 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f2a13658b80 20 client.15474 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f2a13658b80 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e10e310 0x55d74e110790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f2a13658b80 4 client.15474 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:28:20.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.225+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d74e0433a0 con 0x55d74e10e310 2024-05-11T12:28:20.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.235+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e10e310 0x55d74e110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.247+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e10e310 0x55d74e110790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b95d2828408f55a server_cookie=ca686c3d71962263 in_seq=0 out_seq=0 2024-05-11T12:28:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.248+0000 7f29e9ffb700 10 client.15474 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:20.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.253+0000 7f29e9ffb700 1 -- 172.21.15.47:0/948634781 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d74e0433a0 con 0x55d74e10e310 2024-05-11T12:28:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.253+0000 7f29e9ffb700 10 client.15474 handle_command_reply: tid=0 2024-05-11T12:28:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.253+0000 7f2a13658b80 1 client.15474 shutdown 2024-05-11T12:28:20.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.255+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 msgr2=0x55d74e0ef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.255+0000 7f2a13658b80 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0ef5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f29f4004970 tx=0x7f29f4004a50 comp rx=0 tx=0).stop 2024-05-11T12:28:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.262+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 shutdown_connections 2024-05-11T12:28:20.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.262+0000 7f2a13658b80 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e10e310 0x55d74e110790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.262+0000 7f2a13658b80 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e99a0 0x55d74e0ece90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.262+0000 7f2a13658b80 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0ef5b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.262+0000 7f2a13658b80 1 --2- 172.21.15.47:0/948634781 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e0ec950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.262+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 >> 172.21.15.47:0/948634781 conn(0x55d74e00fbf0 msgr2=0x55d74e0dd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.262+0000 7f29e9ffb700 0 client.15474 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:20.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.264+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 shutdown_connections 2024-05-11T12:28:20.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.266+0000 7f2a13658b80 1 -- 172.21.15.47:0/948634781 wait complete. 2024-05-11T12:28:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.269+0000 7f2a13658b80 20 client.15474 trim_cache size 0 max 16384 2024-05-11T12:28:20.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.336+0000 7f2a13658b80 1 Processor -- start 2024-05-11T12:28:20.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.336+0000 7f2a13658b80 1 -- start start 2024-05-11T12:28:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.337+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0e3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.337+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e9080 0x55d74e116060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.337+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e99a0 0x55d74e1165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.338+0000 7f2a13658b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400b830 con 0x55d74e0e3bb0 2024-05-11T12:28:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.338+0000 7f2a13658b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400e0b0 con 0x55d74e0e9080 2024-05-11T12:28:20.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.338+0000 7f2a02d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e99a0 0x55d74e1165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.338+0000 7f2a02d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e99a0 0x55d74e1165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:34540/0 (socket says 172.21.15.47:34540) 2024-05-11T12:28:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.338+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3837266734 learned_addr learned my addr 172.21.15.47:0/3837266734 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.338+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e9080 0x55d74e116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.338+0000 7f2a13658b80 1 -- 172.21.15.47:0/3837266734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400bea0 con 0x55d74e0e99a0 2024-05-11T12:28:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.339+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3837266734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e9080 msgr2=0x55d74e116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.339+0000 7f2a02d9d700 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e9080 0x55d74e116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.339+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3837266734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 msgr2=0x55d74e0e3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.339+0000 7f2a02d9d700 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.339+0000 7f2a02d9d700 1 -- 172.21.15.47:0/3837266734 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d74e0dde60 con 0x55d74e0e99a0 2024-05-11T12:28:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.339+0000 7f2a02d9d700 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e99a0 0x55d74e1165a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0090c0 tx=0x7f29fc0a86b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b25620101f5038be server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.340+0000 7f29ebfff700 1 -- 172.21.15.47:0/3837266734 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0ad1c0 con 0x55d74e0e99a0 2024-05-11T12:28:20.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.340+0000 7f29ebfff700 1 -- 172.21.15.47:0/3837266734 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29fc0b1c70 con 0x55d74e0e99a0 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.341+0000 7f2a13658b80 1 -- 172.21.15.47:0/3837266734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e99a0 msgr2=0x55d74e1165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.341+0000 7f2a13658b80 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e99a0 0x55d74e1165a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0090c0 tx=0x7f29fc0a86b0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.341+0000 7f2a13658b80 1 -- 172.21.15.47:0/3837266734 shutdown_connections 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.341+0000 7f2a13658b80 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e99a0 0x55d74e1165a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.341+0000 7f2a13658b80 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e0e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.341+0000 7f2a13658b80 1 --2- 172.21.15.47:0/3837266734 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e0e9080 0x55d74e116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.341+0000 7f2a13658b80 1 -- 172.21.15.47:0/3837266734 >> 172.21.15.47:0/3837266734 conn(0x55d74e00fbf0 msgr2=0x55d74e0ee890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.342+0000 7f2a13658b80 1 -- 172.21.15.47:0/3837266734 shutdown_connections 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.342+0000 7f2a13658b80 1 -- 172.21.15.47:0/3837266734 wait complete. 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.342+0000 7f2a13658b80 1 -- start start 2024-05-11T12:28:20.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 1 Processor -- start 2024-05-11T12:28:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e104bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e1050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e175c10 0x55d74e178000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400c570 con 0x55d74e0e3bb0 2024-05-11T12:28:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400d9e0 con 0x55d74e175c10 2024-05-11T12:28:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f29e400cc40 con 0x55d74e0e9080 2024-05-11T12:28:20.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.343+0000 7f2a13658b80 10 client.0 _ll_register_callbacks cb 0x55d74dfcd9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.345+0000 7f2a0259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e104bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.345+0000 7f2a0259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e104bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51244/0 (socket says 172.21.15.47:51244) 2024-05-11T12:28:20.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.345+0000 7f2a0259c700 1 -- 172.21.15.47:0/1247031555 learned_addr learned my addr 172.21.15.47:0/1247031555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.346+0000 7f2a0259c700 1 -- 172.21.15.47:0/1247031555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 msgr2=0x55d74e1050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.346+0000 7f2a0259c700 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e1050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.347+0000 7f2a0259c700 1 -- 172.21.15.47:0/1247031555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e175c10 msgr2=0x55d74e178000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.347+0000 7f2a0259c700 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e175c10 0x55d74e178000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.347+0000 7f2a0259c700 1 -- 172.21.15.47:0/1247031555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d74e178540 con 0x55d74e0e3bb0 2024-05-11T12:28:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.347+0000 7f2a0259c700 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e104bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f29f400b2e0 tx=0x7f29f40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddf530daa10fc00 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:20.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.347+0000 7f29e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:28:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.347+0000 7f29e9ffb700 1 -- 172.21.15.47:0/1247031555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f402e040 con 0x55d74e0e3bb0 2024-05-11T12:28:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.348+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d74e03d440 con 0x55d74e0e3bb0 2024-05-11T12:28:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.348+0000 7f29e9ffb700 1 -- 172.21.15.47:0/1247031555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f29f4005150 con 0x55d74e0e3bb0 2024-05-11T12:28:20.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.349+0000 7f29e9ffb700 1 -- 172.21.15.47:0/1247031555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f29f4043050 con 0x55d74e0e3bb0 2024-05-11T12:28:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.349+0000 7f2a13658b80 10 client.15477 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:20.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.349+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d74e1798e0 con 0x55d74e0e3bb0 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.349+0000 7f29e9ffb700 1 -- 172.21.15.47:0/1247031555 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d74e03d440 con 0x55d74e0e3bb0 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.350+0000 7f29e9ffb700 1 -- 172.21.15.47:0/1247031555 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f29f403b020 con 0x55d74e0e3bb0 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.350+0000 7f2a13658b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.350+0000 7f2a13658b80 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.350+0000 7f2a13658b80 20 client.15477 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.350+0000 7f2a13658b80 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e179f00 0x55d74e17c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.350+0000 7f2a13658b80 4 client.15477 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:28:20.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.350+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d74deb52d0 con 0x55d74e179f00 2024-05-11T12:28:20.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.355+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e179f00 0x55d74e17c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.356+0000 7f29e9ffb700 10 client.15477 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:20.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.356+0000 7f2a01d9b700 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e179f00 0x55d74e17c360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff91ec338c35ea6 server_cookie=7cfb1aed9f5f5b3e in_seq=0 out_seq=0 2024-05-11T12:28:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.365+0000 7f29e9ffb700 1 -- 172.21.15.47:0/1247031555 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d74deb52d0 con 0x55d74e179f00 2024-05-11T12:28:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.365+0000 7f29e9ffb700 10 client.15477 handle_command_reply: tid=0 2024-05-11T12:28:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.365+0000 7f2a13658b80 1 client.15477 shutdown 2024-05-11T12:28:20.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.365+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 msgr2=0x55d74e104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.365+0000 7f2a13658b80 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e104bb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f29f400b2e0 tx=0x7f29f40046d0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 shutdown_connections 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d74e179f00 0x55d74e17c360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d74e175c10 0x55d74e178000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d74e0e3bb0 0x55d74e104bb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 --2- 172.21.15.47:0/1247031555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d74e0e9080 0x55d74e1050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 >> 172.21.15.47:0/1247031555 conn(0x55d74e00fbf0 msgr2=0x55d74e0edf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:20.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f29e9ffb700 0 client.15477 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:20.368 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:28:20.368 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:28:20.368 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "e2e69c2b-4c5b-49e9-a3c5-d89ea7f2aed3", 2024-05-11T12:28:20.368 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:28:20.368 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:28:20.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 shutdown_connections 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 1 -- 172.21.15.47:0/1247031555 wait complete. 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.366+0000 7f2a13658b80 20 client.15477 trim_cache size 0 max 16384 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.368+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29ec0b2a10 msgr2=0x7f29ec0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.368+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29ec0b2a10 0x7f29ec0b4ed0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2a04077230 tx=0x7f29f800b040 comp rx=0 tx=0).stop 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.368+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04109550 msgr2=0x7f2a0419c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.368+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04109550 0x7f2a0419c240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f29fc007fd0 tx=0x7f29fc00d3b0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.369+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 shutdown_connections 2024-05-11T12:28:20.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.369+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a04109550 0x7f2a0419c240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.369+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f29ec0b2a10 0x7f29ec0b4ed0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.369+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a041034a0 0x7f2a0410f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.369+0000 7f2a0359e700 1 --2- 172.21.15.47:0/3772067390 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2a04103e60 0x7f2a0410f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:20.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.369+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 >> 172.21.15.47:0/3772067390 conn(0x7f2a040fefd0 msgr2=0x7f2a041003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.372+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 shutdown_connections 2024-05-11T12:28:20.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:20.372+0000 7f2a0359e700 1 -- 172.21.15.47:0/3772067390 wait complete. 2024-05-11T12:28:21.535 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:21 smithi047 ceph-mon[30375]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 750 MiB/s wr, 2.12k op/s 2024-05-11T12:28:21.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:21 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:28:21.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:21 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:21.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:21 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:28:21.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:21 smithi064 ceph-mon[34537]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 750 MiB/s wr, 2.12k op/s 2024-05-11T12:28:21.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:21 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:28:21.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:21 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:21.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:21 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:28:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:21 smithi050 ceph-mon[34072]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 750 MiB/s wr, 2.12k op/s 2024-05-11T12:28:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:21 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:28:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:21 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:21 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:28:22.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:28:22.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:22 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:22.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:22 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:22.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:22 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:23.317 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:28:23.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:23 smithi064 ceph-mon[34537]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 875 MiB/s wr, 2.19k op/s 2024-05-11T12:28:23.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:23 smithi050 ceph-mon[34072]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 875 MiB/s wr, 2.19k op/s 2024-05-11T12:28:23.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:23 smithi047 ceph-mon[30375]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 875 MiB/s wr, 2.19k op/s 2024-05-11T12:28:24.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:24 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:28:24.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:24 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:28:24.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:24 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:28:25.395 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:25 smithi047 ceph-mon[30375]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 2.07k op/s 2024-05-11T12:28:25.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:25 smithi050 ceph-mon[34072]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 2.07k op/s 2024-05-11T12:28:25.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:25 smithi064 ceph-mon[34537]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 856 MiB/s wr, 2.07k op/s 2024-05-11T12:28:27.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:27 smithi050 ceph-mon[34072]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 727 MiB/s wr, 1.70k op/s 2024-05-11T12:28:27.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:27 smithi047 ceph-mon[30375]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 727 MiB/s wr, 1.70k op/s 2024-05-11T12:28:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:27 smithi064 ceph-mon[34537]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 727 MiB/s wr, 1.70k op/s 2024-05-11T12:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.416+0000 7fbd089a5700 1 -- 172.21.15.47:0/1893430673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcfc094f80 msgr2=0x7fbcfc0a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.416+0000 7fbd089a5700 1 --2- 172.21.15.47:0/1893430673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcfc094f80 0x7fbcfc0a65b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbcf801a9a0 tx=0x7fbcf801acb0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.416+0000 7fbd089a5700 1 -- 172.21.15.47:0/1893430673 shutdown_connections 2024-05-11T12:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.416+0000 7fbd089a5700 1 --2- 172.21.15.47:0/1893430673 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 0x7fbcfc0a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.416+0000 7fbd089a5700 1 --2- 172.21.15.47:0/1893430673 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc094660 0x7fbcfc094a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.416+0000 7fbd089a5700 1 --2- 172.21.15.47:0/1893430673 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcfc094f80 0x7fbcfc0a65b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.416+0000 7fbd089a5700 1 -- 172.21.15.47:0/1893430673 >> 172.21.15.47:0/1893430673 conn(0x7fbcfc009f20 msgr2=0x7fbcfc00a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:28.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.421+0000 7fbd089a5700 1 -- 172.21.15.47:0/1893430673 shutdown_connections 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.421+0000 7fbd089a5700 1 -- 172.21.15.47:0/1893430673 wait complete. 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.422+0000 7fbd089a5700 1 Processor -- start 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.422+0000 7fbd089a5700 1 -- start start 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.423+0000 7fbd089a5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc094660 0x7fbcfc0a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.423+0000 7fbd089a5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcfc094f80 0x7fbcfc0a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.423+0000 7fbd089a5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 0x7fbcfc0a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.423+0000 7fbd089a5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc09e670 con 0x7fbcfc094f80 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.423+0000 7fbd089a5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc09e4f0 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.423+0000 7fbd089a5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc09e7f0 con 0x7fbcfc094660 2024-05-11T12:28:28.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.424+0000 7fbd0359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 0x7fbcfc0a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.424+0000 7fbd0359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 0x7fbcfc0a2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:60732/0 (socket says 172.21.15.47:60732) 2024-05-11T12:28:28.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.424+0000 7fbd0359e700 1 -- 172.21.15.47:0/365223359 learned_addr learned my addr 172.21.15.47:0/365223359 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.426+0000 7fbd0359e700 1 -- 172.21.15.47:0/365223359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc094660 msgr2=0x7fbcfc0a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.426+0000 7fbd0359e700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc094660 0x7fbcfc0a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.426+0000 7fbd0359e700 1 -- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcfc094f80 msgr2=0x7fbcfc0a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.426+0000 7fbd0359e700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcfc094f80 0x7fbcfc0a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.426+0000 7fbd0359e700 1 -- 172.21.15.47:0/365223359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc0a2c00 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.427+0000 7fbd0359e700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 0x7fbcfc0a2300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd04051510 tx=0x7fbd04076dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.427+0000 7fbcebfff700 1 -- 172.21.15.47:0/365223359 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04086070 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.427+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc0a2ef0 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.428+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc158d70 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.436+0000 7fbcebfff700 1 -- 172.21.15.47:0/365223359 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd0406cdb0 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.436+0000 7fbcebfff700 1 -- 172.21.15.47:0/365223359 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04080980 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 Processor -- start 2024-05-11T12:28:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 -- start start 2024-05-11T12:28:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62ba0 0x561ef4d66810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d66d50 0x561ef4d67190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561ef4cb4860 con 0x561ef4d62ba0 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561ef4c537c0 con 0x561ef4d66d50 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.439+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561ef4cb0820 con 0x561ef4d62280 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.440+0000 7fbcebfff700 1 -- 172.21.15.47:0/365223359 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbd04089070 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.440+0000 7fbd0259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d66d50 0x561ef4d67190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.440+0000 7fbd0259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d66d50 0x561ef4d67190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:60744/0 (socket says 172.21.15.47:60744) 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.440+0000 7fbd0259c700 1 -- 172.21.15.47:0/646627115 learned_addr learned my addr 172.21.15.47:0/646627115 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.440+0000 7fbd02d9d700 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62ba0 0x561ef4d66810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.441+0000 7fbd0259c700 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.441+0000 7fbd0259c700 1 -- 172.21.15.47:0/646627115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 msgr2=0x561ef4d62660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.441+0000 7fbd0259c700 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d62660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.441+0000 7fbd0259c700 1 -- 172.21.15.47:0/646627115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62ba0 msgr2=0x561ef4d66810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.441+0000 7fbd0259c700 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62ba0 0x561ef4d66810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.441+0000 7fbcebfff700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbcec0b2a90 0x7fbcec0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.441+0000 7fbd0259c700 1 -- 172.21.15.47:0/646627115 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef4c018d0 con 0x561ef4d66d50 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.442+0000 7fbcebfff700 1 -- 172.21.15.47:0/365223359 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fbd0414e850 con 0x7fbcfc0a6af0 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.442+0000 7fbd0259c700 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d66d50 0x561ef4d67190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbcf801c5d0 tx=0x7fbcf801aef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e68c92009363627 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.442+0000 7fbd0259c700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbcec0b2a90 0x7fbcec0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.444+0000 7fbce97fa700 1 -- 172.21.15.47:0/646627115 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf801cc10 con 0x561ef4d66d50 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.444+0000 7fbce97fa700 1 -- 172.21.15.47:0/646627115 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf801cdb0 con 0x561ef4d66d50 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.444+0000 7fbce97fa700 1 -- 172.21.15.47:0/646627115 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80136a0 con 0x561ef4d66d50 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.445+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/646627115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d66d50 msgr2=0x561ef4d67190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.445+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d66d50 0x561ef4d67190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbcf801c5d0 tx=0x7fbcf801aef0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.446+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/646627115 shutdown_connections 2024-05-11T12:28:28.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.446+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d66d50 0x561ef4d67190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.446+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d62660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.446+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/646627115 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62ba0 0x561ef4d66810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.446+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/646627115 >> 172.21.15.47:0/646627115 conn(0x561ef4c8ee60 msgr2=0x561ef4d5ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.447+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/646627115 shutdown_connections 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.447+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/646627115 wait complete. 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.447+0000 7fbd13a3cb80 1 -- start start 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 1 Processor -- start 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 0x561ef4d6e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d6bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561ef4d6efb0 con 0x561ef4d62280 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x561ef4c1ac30 con 0x561ef4d62ba0 2024-05-11T12:28:28.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x561ef4d6f130 con 0x561ef4d66d50 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.448+0000 7fbd13a3cb80 10 client.0 _ll_register_callbacks cb 0x561ef4c4c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.450+0000 7fbd0359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d6bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.450+0000 7fbd0359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d6bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40450/0 (socket says 172.21.15.47:40450) 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.450+0000 7fbd0359e700 1 -- 172.21.15.47:0/2224767020 learned_addr learned my addr 172.21.15.47:0/2224767020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.450+0000 7fbd0259c700 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.450+0000 7fbd02d9d700 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 0x561ef4d6e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.451+0000 7fbd0259c700 1 -- 172.21.15.47:0/2224767020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 msgr2=0x561ef4d6bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.451+0000 7fbd0259c700 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d6bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.451+0000 7fbd0259c700 1 -- 172.21.15.47:0/2224767020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 msgr2=0x561ef4d6e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.451+0000 7fbd0259c700 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 0x561ef4d6e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.451+0000 7fbd0259c700 1 -- 172.21.15.47:0/2224767020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef4c50690 con 0x561ef4d62ba0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.452+0000 7fbd0259c700 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6b600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8006010 tx=0x7fbcf8005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a12a40113514e47 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.453+0000 7fbce6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.453+0000 7fbce6ff5700 1 -- 172.21.15.47:0/2224767020 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80038c0 con 0x561ef4d62ba0 2024-05-11T12:28:28.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.453+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf801cdb0 con 0x561ef4d62ba0 2024-05-11T12:28:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.453+0000 7fbce6ff5700 1 -- 172.21.15.47:0/2224767020 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fbcf8003a60 con 0x561ef4d62ba0 2024-05-11T12:28:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.454+0000 7fbce6ff5700 1 -- 172.21.15.47:0/2224767020 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbcf8042020 con 0x561ef4d62ba0 2024-05-11T12:28:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.455+0000 7fbd0259c700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbcec0b2a90 0x7fbcec0b4f50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbcf801c5d0 tx=0x7fbcf801aef0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.462+0000 7fbd13a3cb80 10 client.25073 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:28.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.462+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561ef4d6efb0 con 0x561ef4d62ba0 2024-05-11T12:28:28.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbce6ff5700 1 -- 172.21.15.47:0/2224767020 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf801cdb0 con 0x561ef4d62ba0 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbce6ff5700 1 -- 172.21.15.47:0/2224767020 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fbcf8032030 con 0x561ef4d62ba0 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbd13a3cb80 10 client.25073 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbd13a3cb80 10 client.25073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbd13a3cb80 20 client.25073 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4d7bf00 0x561ef4d7e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbd13a3cb80 4 client.25073 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.469+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ef4cc2240 con 0x561ef4d7bf00 2024-05-11T12:28:28.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.470+0000 7fbd02d9d700 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4d7bf00 0x561ef4d7e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.471+0000 7fbd02d9d700 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4d7bf00 0x561ef4d7e380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85df9ba69f4b867e server_cookie=f74e97cfb46b21f0 in_seq=0 out_seq=0 2024-05-11T12:28:28.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.471+0000 7fbce6ff5700 10 client.25073 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.475+0000 7fbce6ff5700 1 -- 172.21.15.47:0/2224767020 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561ef4cc2240 con 0x561ef4d7bf00 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.475+0000 7fbce6ff5700 10 client.25073 handle_command_reply: tid=0 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.475+0000 7fbd13a3cb80 1 client.25073 shutdown 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 msgr2=0x561ef4d6b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6b600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8006010 tx=0x7fbcf8005040 comp rx=0 tx=0).stop 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 shutdown_connections 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4d7bf00 0x561ef4d7e380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d6bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 0x561ef4d6e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/2224767020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6b600 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 >> 172.21.15.47:0/2224767020 conn(0x561ef4c8ee60 msgr2=0x561ef4d61090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbce6ff5700 0 client.25073 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 shutdown_connections 2024-05-11T12:28:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.476+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/2224767020 wait complete. 2024-05-11T12:28:28.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.477+0000 7fbd13a3cb80 20 client.25073 trim_cache size 0 max 16384 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.559+0000 7fbd13a3cb80 1 Processor -- start 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.559+0000 7fbd13a3cb80 1 -- start start 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.560+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d6bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.560+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.560+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d66d50 0x561ef4d6e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.561+0000 7fbd02d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d6bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.561+0000 7fbd02d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d6bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40458/0 (socket says 172.21.15.47:40458) 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.561+0000 7fbd02d9d700 1 -- 172.21.15.47:0/1383678235 learned_addr learned my addr 172.21.15.47:0/1383678235 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:28.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.561+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000b800 con 0x561ef4d66d50 2024-05-11T12:28:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.561+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000e080 con 0x561ef4d62ba0 2024-05-11T12:28:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.561+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000be70 con 0x561ef4d62280 2024-05-11T12:28:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.561+0000 7fbd0259c700 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.562+0000 7fbd0359e700 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d66d50 0x561ef4d6e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.563+0000 7fbd0259c700 1 -- 172.21.15.47:0/1383678235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 msgr2=0x561ef4d6bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.563+0000 7fbd0259c700 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d6bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.563+0000 7fbd0259c700 1 -- 172.21.15.47:0/1383678235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d66d50 msgr2=0x561ef4d6e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.563+0000 7fbd0259c700 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d66d50 0x561ef4d6e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.563+0000 7fbd0259c700 1 -- 172.21.15.47:0/1383678235 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ef4d6c500 con 0x561ef4d62ba0 2024-05-11T12:28:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.565+0000 7fbd0259c700 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6dc10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8005f50 tx=0x7fbcf8010460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe05970549e94eca server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.566+0000 7fbce8ff9700 1 -- 172.21.15.47:0/1383678235 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8010b60 con 0x561ef4d62ba0 2024-05-11T12:28:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.566+0000 7fbce8ff9700 1 -- 172.21.15.47:0/1383678235 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf8003d70 con 0x561ef4d62ba0 2024-05-11T12:28:28.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.566+0000 7fbce8ff9700 1 -- 172.21.15.47:0/1383678235 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8013040 con 0x561ef4d62ba0 2024-05-11T12:28:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.567+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 msgr2=0x561ef4d6dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.567+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6dc10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8005f50 tx=0x7fbcf8010460 comp rx=0 tx=0).stop 2024-05-11T12:28:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.568+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 shutdown_connections 2024-05-11T12:28:28.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.568+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d66d50 0x561ef4d6e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.568+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d62280 0x561ef4d6bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.568+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/1383678235 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d6dc10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.568+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 >> 172.21.15.47:0/1383678235 conn(0x561ef4c8ee60 msgr2=0x561ef4d6b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.568+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 shutdown_connections 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.569+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/1383678235 wait complete. 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.569+0000 7fbd13a3cb80 1 -- start start 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.569+0000 7fbd13a3cb80 1 Processor -- start 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.570+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 0x561ef4d74920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.570+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.570+0000 7fbd13a3cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d68630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.570+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000c540 con 0x561ef4d62280 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.570+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000d9b0 con 0x561ef4d62ba0 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.570+0000 7fbd13a3cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000cc10 con 0x561ef4d66d50 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.570+0000 7fbd13a3cb80 10 client.0 _ll_register_callbacks cb 0x561ef4c4c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:28.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.571+0000 7fbd0359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d68630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.571+0000 7fbd0359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d68630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40460/0 (socket says 172.21.15.47:40460) 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.571+0000 7fbd0359e700 1 -- 172.21.15.47:0/129319871 learned_addr learned my addr 172.21.15.47:0/129319871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.572+0000 7fbd0259c700 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.572+0000 7fbd0359e700 1 -- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 msgr2=0x561ef4d680f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.572+0000 7fbd0359e700 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d680f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.572+0000 7fbd0359e700 1 -- 172.21.15.47:0/129319871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 msgr2=0x561ef4d74920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.572+0000 7fbd0359e700 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 0x561ef4d74920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.572+0000 7fbd0359e700 1 -- 172.21.15.47:0/129319871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ef4d68d70 con 0x561ef4d66d50 2024-05-11T12:28:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.575+0000 7fbd0359e700 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d68630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbd04115d60 tx=0x7fbd04115a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da78c8adf59e7ce2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.575+0000 7fbce6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:28:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.575+0000 7fbce6ff5700 1 -- 172.21.15.47:0/129319871 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04119040 con 0x561ef4d66d50 2024-05-11T12:28:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.576+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf8003d70 con 0x561ef4d66d50 2024-05-11T12:28:28.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.576+0000 7fbce6ff5700 1 -- 172.21.15.47:0/129319871 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fbd04115240 con 0x561ef4d66d50 2024-05-11T12:28:28.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.576+0000 7fbce6ff5700 1 -- 172.21.15.47:0/129319871 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fbd04115dd0 con 0x561ef4d66d50 2024-05-11T12:28:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.578+0000 7fbd13a3cb80 10 client.25114 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:28.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.578+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561ef4df77b0 con 0x561ef4d66d50 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.580+0000 7fbce6ff5700 1 -- 172.21.15.47:0/129319871 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf8003d70 con 0x561ef4d66d50 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.580+0000 7fbce6ff5700 1 -- 172.21.15.47:0/129319871 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fbd04111740 con 0x561ef4d66d50 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.582+0000 7fbd13a3cb80 10 client.25114 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.582+0000 7fbd13a3cb80 10 client.25114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.582+0000 7fbd13a3cb80 20 client.25114 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.582+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4df7ec0 0x561ef4dfa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.582+0000 7fbd13a3cb80 4 client.25114 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:28:28.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.582+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ef4cbc2e0 con 0x561ef4df7ec0 2024-05-11T12:28:28.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.583+0000 7fbd02d9d700 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4df7ec0 0x561ef4dfa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.587+0000 7fbd02d9d700 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4df7ec0 0x561ef4dfa2e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8c7627a1120518 server_cookie=41e782cc84dc83d in_seq=0 out_seq=0 2024-05-11T12:28:28.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.587+0000 7fbce6ff5700 10 client.25114 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.594+0000 7fbce6ff5700 1 -- 172.21.15.47:0/129319871 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561ef4cbc2e0 con 0x561ef4df7ec0 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.594+0000 7fbce6ff5700 10 client.25114 handle_command_reply: tid=0 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.594+0000 7fbd13a3cb80 1 client.25114 shutdown 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.594+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 msgr2=0x561ef4d68630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.594+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d68630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbd04115d60 tx=0x7fbd04115a40 comp rx=0 tx=0).stop 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 shutdown_connections 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x561ef4df7ec0 0x561ef4dfa2e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x561ef4d66d50 0x561ef4d68630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561ef4d62280 0x561ef4d74920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbd13a3cb80 1 --2- 172.21.15.47:0/129319871 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x561ef4d62ba0 0x561ef4d680f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 >> 172.21.15.47:0/129319871 conn(0x561ef4c8ee60 msgr2=0x561ef4d6a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbce6ff5700 0 client.25114 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.595+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 shutdown_connections 2024-05-11T12:28:28.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.596+0000 7fbd13a3cb80 1 -- 172.21.15.47:0/129319871 wait complete. 2024-05-11T12:28:28.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.596+0000 7fbd13a3cb80 20 client.25114 trim_cache size 0 max 16384 2024-05-11T12:28:28.600 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:28:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:28:28.600 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:28:28.600 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbcec0b2a90 msgr2=0x7fbcec0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbcec0b2a90 0x7fbcec0b4f50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbcf801c5d0 tx=0x7fbcf801aef0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 msgr2=0x7fbcfc0a2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 0x7fbcfc0a2300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd04051510 tx=0x7fbd04076dd0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 shutdown_connections 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fbcec0b2a90 0x7fbcec0b4f50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fbcfc0a6af0 0x7fbcfc0a2300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fbcfc094660 0x7fbcfc0a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 --2- 172.21.15.47:0/365223359 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbcfc094f80 0x7fbcfc0a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:28.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.600+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 >> 172.21.15.47:0/365223359 conn(0x7fbcfc009f20 msgr2=0x7fbcfc0a70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.601+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 shutdown_connections 2024-05-11T12:28:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:28.601+0000 7fbd089a5700 1 -- 172.21.15.47:0/365223359 wait complete. 2024-05-11T12:28:29.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:29 smithi050 ceph-mon[34072]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 893 MiB/s wr, 2.17k op/s 2024-05-11T12:28:29.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:29 smithi047 ceph-mon[30375]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 893 MiB/s wr, 2.17k op/s 2024-05-11T12:28:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:29 smithi064 ceph-mon[34537]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 893 MiB/s wr, 2.17k op/s 2024-05-11T12:28:30.465 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:28:30.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:28:30.596 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:30 smithi047 ceph-mon[30375]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.69k op/s 2024-05-11T12:28:30.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:30 smithi064 ceph-mon[34537]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.69k op/s 2024-05-11T12:28:30.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:30 smithi050 ceph-mon[34072]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 763 MiB/s wr, 1.69k op/s 2024-05-11T12:28:31.569 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:28:33.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:32 smithi050 ceph-mon[34072]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-05-11T12:28:33.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:32 smithi047 ceph-mon[30375]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-05-11T12:28:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:32 smithi064 ceph-mon[34537]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 890 MiB/s wr, 1.75k op/s 2024-05-11T12:28:35.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:34 smithi047 ceph-mon[30375]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 826 MiB/s wr, 1.75k op/s 2024-05-11T12:28:35.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:34 smithi050 ceph-mon[34072]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 826 MiB/s wr, 1.75k op/s 2024-05-11T12:28:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:34 smithi064 ceph-mon[34537]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 826 MiB/s wr, 1.75k op/s 2024-05-11T12:28:37.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:36 smithi047 ceph-mon[30375]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 735 MiB/s wr, 1.62k op/s 2024-05-11T12:28:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:36 smithi064 ceph-mon[34537]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 735 MiB/s wr, 1.62k op/s 2024-05-11T12:28:37.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:36 smithi050 ceph-mon[34072]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 735 MiB/s wr, 1.62k op/s 2024-05-11T12:28:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.632+0000 7fdfedfe9700 1 -- 172.21.15.47:0/376449014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe81060f0 msgr2=0x7fdfe81065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.632+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/376449014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe81060f0 0x7fdfe81065a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fdfe000b0a0 tx=0x7fdfe000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:28:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.634+0000 7fdfedfe9700 1 -- 172.21.15.47:0/376449014 shutdown_connections 2024-05-11T12:28:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.634+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/376449014 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe81060f0 0x7fdfe81065a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.634+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/376449014 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfe8108650 0x7fdfe806bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.634+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/376449014 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe806c140 0x7fdfe806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.634+0000 7fdfedfe9700 1 -- 172.21.15.47:0/376449014 >> 172.21.15.47:0/376449014 conn(0x7fdfe8079250 msgr2=0x7fdfe8079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.638+0000 7fdfedfe9700 1 -- 172.21.15.47:0/376449014 shutdown_connections 2024-05-11T12:28:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.639+0000 7fdfedfe9700 1 -- 172.21.15.47:0/376449014 wait complete. 2024-05-11T12:28:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.641+0000 7fdfedfe9700 1 Processor -- start 2024-05-11T12:28:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.641+0000 7fdfedfe9700 1 -- start start 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.642+0000 7fdfedfe9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe806c140 0x7fdfe81a0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.642+0000 7fdfedfe9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfe81060f0 0x7fdfe81a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.642+0000 7fdfedfe9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8108650 0x7fdfe81a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.642+0000 7fdfedfe9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8119e10 con 0x7fdfe8108650 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.643+0000 7fdfedfe9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8119c90 con 0x7fdfe81060f0 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.643+0000 7fdfedfe9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe8119f90 con 0x7fdfe806c140 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.643+0000 7fdfe77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe806c140 0x7fdfe81a0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.643+0000 7fdfe77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe806c140 0x7fdfe81a0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52132/0 (socket says 172.21.15.47:52132) 2024-05-11T12:28:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.643+0000 7fdfe77fe700 1 -- 172.21.15.47:0/2104191755 learned_addr learned my addr 172.21.15.47:0/2104191755 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.643+0000 7fdfe6ffd700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfe81060f0 0x7fdfe81a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.644+0000 7fdfe6ffd700 1 -- 172.21.15.47:0/2104191755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe806c140 msgr2=0x7fdfe81a0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.644+0000 7fdfe6ffd700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe806c140 0x7fdfe81a0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.644+0000 7fdfe6ffd700 1 -- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8108650 msgr2=0x7fdfe81a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.644+0000 7fdfe6ffd700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8108650 0x7fdfe81a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.645+0000 7fdfe6ffd700 1 -- 172.21.15.47:0/2104191755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe81a5210 con 0x7fdfe81060f0 2024-05-11T12:28:37.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.645+0000 7fdfe6ffd700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfe81060f0 0x7fdfe81a0740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00d950 tx=0x7fdfdc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.647+0000 7fdfe4ff9700 1 -- 172.21.15.47:0/2104191755 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc017070 con 0x7fdfe81060f0 2024-05-11T12:28:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.647+0000 7fdfe4ff9700 1 -- 172.21.15.47:0/2104191755 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfdc009d60 con 0x7fdfe81060f0 2024-05-11T12:28:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.647+0000 7fdfe4ff9700 1 -- 172.21.15.47:0/2104191755 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc010d90 con 0x7fdfe81060f0 2024-05-11T12:28:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.647+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe81a5500 con 0x7fdfe81060f0 2024-05-11T12:28:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.647+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe81acd80 con 0x7fdfe81060f0 2024-05-11T12:28:37.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.650+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4005330 con 0x7fdfe81060f0 2024-05-11T12:28:37.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.653+0000 7fdfe4ff9700 1 -- 172.21.15.47:0/2104191755 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdfdc01a070 con 0x7fdfe81060f0 2024-05-11T12:28:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.654+0000 7fdfe4ff9700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdfd00b2a10 0x7fdfd00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.655+0000 7fdfe4ff9700 1 -- 172.21.15.47:0/2104191755 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fdfdc0e1aa0 con 0x7fdfe81060f0 2024-05-11T12:28:37.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.656+0000 7fdfe77fe700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdfd00b2a10 0x7fdfd00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:37.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.658+0000 7fdfe77fe700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdfd00b2a10 0x7fdfd00b4ed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8005950 tx=0x7fdfd800a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:37.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:37.661+0000 7fdfe4ff9700 1 -- 172.21.15.47:0/2104191755 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdfdc0a97f0 con 0x7fdfe81060f0 2024-05-11T12:28:38.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:37 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:28:38.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.317+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfd4005620 con 0x7fdfe81060f0 2024-05-11T12:28:38.323 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:28:38.323 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:28:38.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.322+0000 7fdfe4ff9700 1 -- 172.21.15.47:0/2104191755 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fdfdc0a97f0 con 0x7fdfe81060f0 2024-05-11T12:28:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdfd00b2a10 msgr2=0x7fdfd00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdfd00b2a10 0x7fdfd00b4ed0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8005950 tx=0x7fdfd800a400 comp rx=0 tx=0).stop 2024-05-11T12:28:38.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfe81060f0 msgr2=0x7fdfe81a0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfe81060f0 0x7fdfe81a0740 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00d950 tx=0x7fdfdc00dc60 comp rx=0 tx=0).stop 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 shutdown_connections 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdfe8108650 0x7fdfe81a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdfe806c140 0x7fdfe81a0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdfd00b2a10 0x7fdfd00b4ed0 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8005950 tx=0x7fdfd800a400 comp rx=0 tx=0).stop 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 --2- 172.21.15.47:0/2104191755 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdfe81060f0 0x7fdfe81a0740 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00d950 tx=0x7fdfdc00dc60 comp rx=0 tx=0).stop 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.336+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 >> 172.21.15.47:0/2104191755 conn(0x7fdfe8079250 msgr2=0x7fdfe8101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.337+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 shutdown_connections 2024-05-11T12:28:38.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:38.337+0000 7fdfedfe9700 1 -- 172.21.15.47:0/2104191755 wait complete. 2024-05-11T12:28:38.340 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:28:38.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:37 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:28:38.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:37 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:28:39.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:38 smithi047 ceph-mon[30375]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 912 MiB/s wr, 1.94k op/s 2024-05-11T12:28:39.042 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:38 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2104191755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:28:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:38 smithi064 ceph-mon[34537]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 912 MiB/s wr, 1.94k op/s 2024-05-11T12:28:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:38 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2104191755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:28:39.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:38 smithi050 ceph-mon[34072]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 912 MiB/s wr, 1.94k op/s 2024-05-11T12:28:39.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:38 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2104191755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:28:40.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:28:41.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:41 smithi047 ceph-mon[30375]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 MiB/s wr, 1.41k op/s 2024-05-11T12:28:41.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:28:41.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:28:41.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:28:41.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:41 smithi064 ceph-mon[34537]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 MiB/s wr, 1.41k op/s 2024-05-11T12:28:41.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:28:41.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:28:41.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:28:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:28:41.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:41 smithi050 ceph-mon[34072]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 MiB/s wr, 1.41k op/s 2024-05-11T12:28:41.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:28:41.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:28:41.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:28:43.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:43 smithi047 ceph-mon[30375]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.60k op/s 2024-05-11T12:28:43.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:43 smithi064 ceph-mon[34537]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.60k op/s 2024-05-11T12:28:43.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:43 smithi050 ceph-mon[34072]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.60k op/s 2024-05-11T12:28:45.338 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:45 smithi047 ceph-mon[30375]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 752 MiB/s wr, 1.58k op/s 2024-05-11T12:28:45.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:45 smithi064 ceph-mon[34537]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 752 MiB/s wr, 1.58k op/s 2024-05-11T12:28:45.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:45 smithi050 ceph-mon[34072]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 752 MiB/s wr, 1.58k op/s 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 -- 172.21.15.47:0/2797709152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd370111d30 msgr2=0x7fd37010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 --2- 172.21.15.47:0/2797709152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd370111d30 0x7fd37010a550 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd36801c0d0 tx=0x7fd36801c3e0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 -- 172.21.15.47:0/2797709152 shutdown_connections 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 --2- 172.21.15.47:0/2797709152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd37011cd30 0x7fd37011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 --2- 172.21.15.47:0/2797709152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd370111d30 0x7fd37010a550 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 --2- 172.21.15.47:0/2797709152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd37010aa90 0x7fd37010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 -- 172.21.15.47:0/2797709152 >> 172.21.15.47:0/2797709152 conn(0x7fd37006e6c0 msgr2=0x7fd37006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 -- 172.21.15.47:0/2797709152 shutdown_connections 2024-05-11T12:28:46.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.866+0000 7fd374ca6700 1 -- 172.21.15.47:0/2797709152 wait complete. 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 Processor -- start 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 -- start start 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd37010aa90 0x7fd370118450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370111d30 0x7fd370118990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd37011cd30 0x7fd37011bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd370114d30 con 0x7fd37010aa90 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd370114bb0 con 0x7fd37011cd30 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.868+0000 7fd374ca6700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd370114eb0 con 0x7fd370111d30 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36ed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd37011cd30 0x7fd37011bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36ed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd37011cd30 0x7fd37011bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:37100/0 (socket says 172.21.15.47:37100) 2024-05-11T12:28:46.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36ed9d700 1 -- 172.21.15.47:0/1441651432 learned_addr learned my addr 172.21.15.47:0/1441651432 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370111d30 0x7fd370118990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36dd9b700 1 -- 172.21.15.47:0/1441651432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd37011cd30 msgr2=0x7fd37011bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd37011cd30 0x7fd37011bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36dd9b700 1 -- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd37010aa90 msgr2=0x7fd370118450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd37010aa90 0x7fd370118450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36dd9b700 1 -- 172.21.15.47:0/1441651432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37011c450 con 0x7fd370111d30 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.869+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370111d30 0x7fd370118990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd36400d950 tx=0x7fd36400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.870+0000 7fd35f7fe700 1 -- 172.21.15.47:0/1441651432 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364017070 con 0x7fd370111d30 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.870+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37011c740 con 0x7fd370111d30 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.870+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3701ace30 con 0x7fd370111d30 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.871+0000 7fd35f7fe700 1 -- 172.21.15.47:0/1441651432 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd364009de0 con 0x7fd370111d30 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.871+0000 7fd35f7fe700 1 -- 172.21.15.47:0/1441651432 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd364010e20 con 0x7fd370111d30 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.878+0000 7fd35f7fe700 1 -- 172.21.15.47:0/1441651432 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd36401a070 con 0x7fd370111d30 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.880+0000 7fd37f53cb80 1 Processor -- start 2024-05-11T12:28:46.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.880+0000 7fd37f53cb80 1 -- start start 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.880+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a42eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.880+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a3d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.880+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a49890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5580d198f860 con 0x5580d1a42ad0 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5580d192e7c0 con 0x5580d1a3d0a0 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5580d198b820 con 0x5580d1a3da60 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd35f7fe700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3580b2a90 0x7fd3580b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd36ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a42eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd36ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a42eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47206/0 (socket says 172.21.15.47:47206) 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd36ed9d700 1 -- 172.21.15.47:0/1186513123 learned_addr learned my addr 172.21.15.47:0/1186513123 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd35f7fe700 1 -- 172.21.15.47:0/1441651432 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd3640e1b50 con 0x7fd370111d30 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.881+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3580b2a90 0x7fd3580b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.882+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a49890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.882+0000 7fd36ed9d700 1 -- 172.21.15.47:0/1186513123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 msgr2=0x5580d1a49890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.882+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a49890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.882+0000 7fd36ed9d700 1 -- 172.21.15.47:0/1186513123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 msgr2=0x5580d1a3d520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.882+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a3d520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.882+0000 7fd36ed9d700 1 -- 172.21.15.47:0/1186513123 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d18dc8d0 con 0x5580d1a42ad0 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.886+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a42eb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd360012840 tx=0x7fd360012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ee5e73713e18b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.887+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3580b2a90 0x7fd3580b4f50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd360012fd0 tx=0x7fd36001d000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.891+0000 7fd35cff9700 1 -- 172.21.15.47:0/1186513123 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360023070 con 0x5580d1a42ad0 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.891+0000 7fd35cff9700 1 -- 172.21.15.47:0/1186513123 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd36000ee30 con 0x5580d1a42ad0 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.892+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1186513123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 msgr2=0x5580d1a42eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.892+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a42eb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd360012840 tx=0x7fd360012b50 comp rx=0 tx=0).stop 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.896+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1186513123 shutdown_connections 2024-05-11T12:28:46.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.896+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a49890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.896+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a42eb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.896+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1186513123 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a3d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.896+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1186513123 >> 172.21.15.47:0/1186513123 conn(0x5580d1969e60 msgr2=0x5580d1a37990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.899+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1186513123 shutdown_connections 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.899+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1186513123 wait complete. 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.899+0000 7fd37f53cb80 1 -- start start 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 1 Processor -- start 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3d0a0 0x5580d1a494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a3da60 0x5580d1a46860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a42ad0 0x5580d1a46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5580d1a49e30 con 0x5580d1a3da60 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5580d18f5c30 con 0x5580d1a42ad0 2024-05-11T12:28:46.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5580d1a49fb0 con 0x5580d1a3d0a0 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.900+0000 7fd37f53cb80 10 client.0 _ll_register_callbacks cb 0x5580d1927920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.904+0000 7fd36e59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3d0a0 0x5580d1a494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.904+0000 7fd36ed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a42ad0 0x5580d1a46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.904+0000 7fd36dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a3da60 0x5580d1a46860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.904+0000 7fd36dd9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a3da60 0x5580d1a46860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47210/0 (socket says 172.21.15.47:47210) 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.904+0000 7fd36dd9b700 1 -- 172.21.15.47:0/2120630643 learned_addr learned my addr 172.21.15.47:0/2120630643 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.907+0000 7fd36e59c700 1 -- 172.21.15.47:0/2120630643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a42ad0 msgr2=0x5580d1a46da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.907+0000 7fd36e59c700 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a42ad0 0x5580d1a46da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.907+0000 7fd36e59c700 1 -- 172.21.15.47:0/2120630643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a3da60 msgr2=0x5580d1a46860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.907+0000 7fd36e59c700 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a3da60 0x5580d1a46860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.907+0000 7fd36e59c700 1 -- 172.21.15.47:0/2120630643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d192b690 con 0x5580d1a3d0a0 2024-05-11T12:28:46.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.912+0000 7fd36e59c700 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3d0a0 0x5580d1a494c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd36801c860 tx=0x7fd368004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b7b63d695fe40c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:46.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.912+0000 7fd355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:28:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.914+0000 7fd355ffb700 1 -- 172.21.15.47:0/2120630643 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3680078b0 con 0x5580d1a3d0a0 2024-05-11T12:28:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.914+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580d199d240 con 0x5580d1a3d0a0 2024-05-11T12:28:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.915+0000 7fd355ffb700 1 -- 172.21.15.47:0/2120630643 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd368007a50 con 0x5580d1a3d0a0 2024-05-11T12:28:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.915+0000 7fd355ffb700 1 -- 172.21.15.47:0/2120630643 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd36801acd0 con 0x5580d1a3d0a0 2024-05-11T12:28:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.915+0000 7fd37f53cb80 10 client.25132 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.915+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5580d1a767d0 con 0x5580d1a3d0a0 2024-05-11T12:28:46.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.916+0000 7fd355ffb700 1 -- 172.21.15.47:0/2120630643 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5580d199d240 con 0x5580d1a3d0a0 2024-05-11T12:28:46.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.917+0000 7fd355ffb700 1 -- 172.21.15.47:0/2120630643 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd36800f070 con 0x5580d1a3d0a0 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.918+0000 7fd37f53cb80 10 client.25132 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.918+0000 7fd37f53cb80 10 client.25132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.918+0000 7fd37f53cb80 20 client.25132 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.918+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1a79260 0x5580d1a7b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.918+0000 7fd37f53cb80 4 client.25132 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.918+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580d19972e0 con 0x5580d1a79260 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.919+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1a79260 0x5580d1a7b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.919+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1a79260 0x5580d1a7b6a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93e328fdfdd18d4 server_cookie=41d9dd640545750d in_seq=0 out_seq=0 2024-05-11T12:28:46.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.920+0000 7fd355ffb700 10 client.25132 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.920+0000 7fd355ffb700 1 -- 172.21.15.47:0/2120630643 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5580d19972e0 con 0x5580d1a79260 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.920+0000 7fd355ffb700 10 client.25132 handle_command_reply: tid=0 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.920+0000 7fd37f53cb80 1 client.25132 shutdown 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3d0a0 msgr2=0x5580d1a494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3d0a0 0x5580d1a494c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd36801c860 tx=0x7fd368004c40 comp rx=0 tx=0).stop 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 shutdown_connections 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1a79260 0x5580d1a7b6a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a42ad0 0x5580d1a46da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3d0a0 0x5580d1a494c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/2120630643 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a3da60 0x5580d1a46860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.921+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 >> 172.21.15.47:0/2120630643 conn(0x5580d1969e60 msgr2=0x5580d1a32460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.922+0000 7fd355ffb700 0 client.25132 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.922+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 shutdown_connections 2024-05-11T12:28:46.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.922+0000 7fd37f53cb80 1 -- 172.21.15.47:0/2120630643 wait complete. 2024-05-11T12:28:46.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.923+0000 7fd37f53cb80 20 client.25132 trim_cache size 0 max 16384 2024-05-11T12:28:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.961+0000 7fd37f53cb80 1 Processor -- start 2024-05-11T12:28:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.961+0000 7fd37f53cb80 1 -- start start 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.961+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a52fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.961+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a3c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.961+0000 7fd36e59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a52fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.961+0000 7fd36e59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a52fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:37120/0 (socket says 172.21.15.47:37120) 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.961+0000 7fd36e59c700 1 -- 172.21.15.47:0/1561314141 learned_addr learned my addr 172.21.15.47:0/1561314141 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.962+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a3c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.962+0000 7fd36dd9b700 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a3c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.962+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd35000b830 con 0x5580d1a42ad0 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.962+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd35000e0b0 con 0x5580d1a3d0a0 2024-05-11T12:28:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.962+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd35000bea0 con 0x5580d1a3da60 2024-05-11T12:28:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.962+0000 7fd36e59c700 1 -- 172.21.15.47:0/1561314141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 msgr2=0x5580d1a3c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.963+0000 7fd36e59c700 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a3c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.963+0000 7fd36e59c700 1 -- 172.21.15.47:0/1561314141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 msgr2=0x5580d1a3c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.963+0000 7fd36e59c700 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a3c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.963+0000 7fd36e59c700 1 -- 172.21.15.47:0/1561314141 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d1a38560 con 0x5580d1a3d0a0 2024-05-11T12:28:46.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.963+0000 7fd36e59c700 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a52fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd368011db0 tx=0x7fd36801a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b499ef71fa095365 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.967+0000 7fd357fff700 1 -- 172.21.15.47:0/1561314141 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd368005c60 con 0x5580d1a3d0a0 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.967+0000 7fd357fff700 1 -- 172.21.15.47:0/1561314141 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd368005e00 con 0x5580d1a3d0a0 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 msgr2=0x5580d1a52fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a52fa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd368011db0 tx=0x7fd36801a740 comp rx=0 tx=0).stop 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 shutdown_connections 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a3c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a3d0a0 0x5580d1a52fa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/1561314141 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a3c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 >> 172.21.15.47:0/1561314141 conn(0x5580d1969e60 msgr2=0x5580d1a7b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 shutdown_connections 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.968+0000 7fd37f53cb80 1 -- 172.21.15.47:0/1561314141 wait complete. 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.969+0000 7fd37f53cb80 1 -- start start 2024-05-11T12:28:46.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.969+0000 7fd37f53cb80 1 Processor -- start 2024-05-11T12:28:46.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.970+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a4fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.970+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.970+0000 7fd37f53cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a4c960 0x5580d1ae1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd35000c570 con 0x5580d1a42ad0 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd35000d9e0 con 0x5580d1a4c960 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd37f53cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd35000cc40 con 0x5580d1a3da60 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd36ed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a4c960 0x5580d1ae1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd36ed9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a4c960 0x5580d1ae1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:37124/0 (socket says 172.21.15.47:37124) 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd36ed9d700 1 -- 172.21.15.47:0/3822391893 learned_addr learned my addr 172.21.15.47:0/3822391893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd37f53cb80 10 client.0 _ll_register_callbacks cb 0x5580d1927920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.971+0000 7fd36e59c700 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a4fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.972+0000 7fd36ed9d700 1 -- 172.21.15.47:0/3822391893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 msgr2=0x5580d1a4fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.972+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a4fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.972+0000 7fd36ed9d700 1 -- 172.21.15.47:0/3822391893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 msgr2=0x5580d1a4c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.972+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.972+0000 7fd36ed9d700 1 -- 172.21.15.47:0/3822391893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d1a4d0e0 con 0x5580d1a4c960 2024-05-11T12:28:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.972+0000 7fd36ed9d700 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a4c960 0x5580d1ae1b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd360012840 tx=0x7fd360020740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bfe23ee63df91f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:46.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.972+0000 7fd355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.973+0000 7fd355ffb700 1 -- 172.21.15.47:0/3822391893 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360023040 con 0x5580d1a4c960 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.973+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580d180f250 con 0x5580d1a4c960 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.973+0000 7fd355ffb700 1 -- 172.21.15.47:0/3822391893 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd360020df0 con 0x5580d1a4c960 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.974+0000 7fd355ffb700 1 -- 172.21.15.47:0/3822391893 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd36000ee30 con 0x5580d1a4c960 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.974+0000 7fd37f53cb80 10 client.25097 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.974+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5580d1ae3160 con 0x5580d1a4c960 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd355ffb700 1 -- 172.21.15.47:0/3822391893 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5580d180f250 con 0x5580d1a4c960 2024-05-11T12:28:46.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd355ffb700 1 -- 172.21.15.47:0/3822391893 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd360027ae0 con 0x5580d1a4c960 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd37f53cb80 10 client.25097 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd37f53cb80 10 client.25097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd37f53cb80 20 client.25097 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1ae3760 0x5580d1ae5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd37f53cb80 4 client.25097 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580d195fd20 con 0x5580d1ae3760 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.975+0000 7fd36e59c700 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1ae3760 0x5580d1ae5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.976+0000 7fd36e59c700 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1ae3760 0x5580d1ae5b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=392fde9386bc8f18 server_cookie=681917e66e7d8f27 in_seq=0 out_seq=0 2024-05-11T12:28:46.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.976+0000 7fd355ffb700 10 client.25097 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.977+0000 7fd355ffb700 1 -- 172.21.15.47:0/3822391893 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580d195fd20 con 0x5580d1ae3760 2024-05-11T12:28:46.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.977+0000 7fd355ffb700 10 client.25097 handle_command_reply: tid=0 2024-05-11T12:28:46.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.979+0000 7fd37f53cb80 1 client.25097 shutdown 2024-05-11T12:28:46.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a4c960 msgr2=0x5580d1ae1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a4c960 0x5580d1ae1b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd360012840 tx=0x7fd360020740 comp rx=0 tx=0).stop 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 shutdown_connections 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5580d1ae3760 0x5580d1ae5b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5580d1a4c960 0x5580d1ae1b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5580d1a3da60 0x5580d1a4fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 --2- 172.21.15.47:0/3822391893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5580d1a42ad0 0x5580d1a4c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 >> 172.21.15.47:0/3822391893 conn(0x5580d1969e60 msgr2=0x5580d1a31bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:46.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd355ffb700 0 client.25097 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:46.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.986+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 shutdown_connections 2024-05-11T12:28:46.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.987+0000 7fd37f53cb80 1 -- 172.21.15.47:0/3822391893 wait complete. 2024-05-11T12:28:46.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.987+0000 7fd37f53cb80 20 client.25097 trim_cache size 0 max 16384 2024-05-11T12:28:46.990 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3580b2a90 msgr2=0x7fd3580b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3580b2a90 0x7fd3580b4f50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd360012fd0 tx=0x7fd36001d000 comp rx=0 tx=0).stop 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370111d30 msgr2=0x7fd370118990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370111d30 0x7fd370118990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd36400d950 tx=0x7fd36400dc60 comp rx=0 tx=0).stop 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 shutdown_connections 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3580b2a90 0x7fd3580b4f50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd37011cd30 0x7fd37011bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd37010aa90 0x7fd370118450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 --2- 172.21.15.47:0/1441651432 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd370111d30 0x7fd370118990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:46.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.990+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 >> 172.21.15.47:0/1441651432 conn(0x7fd37006e6c0 msgr2=0x7fd37011e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.994+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 shutdown_connections 2024-05-11T12:28:46.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:46.994+0000 7fd374ca6700 1 -- 172.21.15.47:0/1441651432 wait complete. 2024-05-11T12:28:47.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:47 smithi047 ceph-mon[30375]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 729 MiB/s wr, 1.67k op/s 2024-05-11T12:28:47.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:47 smithi064 ceph-mon[34537]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 729 MiB/s wr, 1.67k op/s 2024-05-11T12:28:47.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:47 smithi050 ceph-mon[34072]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 729 MiB/s wr, 1.67k op/s 2024-05-11T12:28:49.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:49 smithi064 ceph-mon[34537]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 888 MiB/s wr, 2.04k op/s 2024-05-11T12:28:49.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:49 smithi050 ceph-mon[34072]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 888 MiB/s wr, 2.04k op/s 2024-05-11T12:28:49.514 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:49 smithi047 ceph-mon[30375]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 888 MiB/s wr, 2.04k op/s 2024-05-11T12:28:49.670 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:28:49.670 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:28:50.578 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:28:51.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:51 smithi064 ceph-mon[34537]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 732 MiB/s wr, 1.75k op/s 2024-05-11T12:28:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:51 smithi050 ceph-mon[34072]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 732 MiB/s wr, 1.75k op/s 2024-05-11T12:28:51.485 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:51 smithi047 ceph-mon[30375]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 732 MiB/s wr, 1.75k op/s 2024-05-11T12:28:53.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:53 smithi047 ceph-mon[30375]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 806 MiB/s wr, 1.97k op/s 2024-05-11T12:28:53.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:53 smithi064 ceph-mon[34537]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 806 MiB/s wr, 1.97k op/s 2024-05-11T12:28:53.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:53 smithi050 ceph-mon[34072]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 806 MiB/s wr, 1.97k op/s 2024-05-11T12:28:55.436 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:55 smithi047 ceph-mon[30375]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 737 MiB/s wr, 1.72k op/s 2024-05-11T12:28:55.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:55 smithi050 ceph-mon[34072]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 737 MiB/s wr, 1.72k op/s 2024-05-11T12:28:55.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:55 smithi064 ceph-mon[34537]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 737 MiB/s wr, 1.72k op/s 2024-05-11T12:28:56.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.032+0000 7fb07b3d1700 1 -- 172.21.15.47:0/280719658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0741077e0 msgr2=0x7fb07410f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.032+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/280719658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0741077e0 0x7fb07410f380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb06c01c0d0 tx=0x7fb06c01c3e0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.034+0000 7fb07b3d1700 1 -- 172.21.15.47:0/280719658 shutdown_connections 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.034+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/280719658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb0741077e0 0x7fb07410f380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.034+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/280719658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb074102a90 0x7fb074102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.034+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/280719658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074103440 0x7fb0741070b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.034+0000 7fb07b3d1700 1 -- 172.21.15.47:0/280719658 >> 172.21.15.47:0/280719658 conn(0x7fb074079400 msgr2=0x7fb074079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.035+0000 7fb07b3d1700 1 -- 172.21.15.47:0/280719658 shutdown_connections 2024-05-11T12:28:56.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.035+0000 7fb07b3d1700 1 -- 172.21.15.47:0/280719658 wait complete. 2024-05-11T12:28:56.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.039+0000 7fb07b3d1700 1 Processor -- start 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.039+0000 7fb07b3d1700 1 -- start start 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.040+0000 7fb07b3d1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074102a90 0x7fb07410ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.041+0000 7fb07b3d1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb074103440 0x7fb07410b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.041+0000 7fb07916d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074102a90 0x7fb07410ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.041+0000 7fb07916d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074102a90 0x7fb07410ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47240/0 (socket says 172.21.15.47:47240) 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.041+0000 7fb07916d700 1 -- 172.21.15.47:0/730858010 learned_addr learned my addr 172.21.15.47:0/730858010 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.041+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0741077e0 0x7fb074195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.041+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406f740 con 0x7fb074102a90 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.041+0000 7fb07896c700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb074103440 0x7fb07410b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.042+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406f5c0 con 0x7fb074103440 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.042+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb07406f8c0 con 0x7fb0741077e0 2024-05-11T12:28:56.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.042+0000 7fb07996e700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0741077e0 0x7fb074195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.046+0000 7fb07916d700 1 -- 172.21.15.47:0/730858010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0741077e0 msgr2=0x7fb074195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.046+0000 7fb07916d700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0741077e0 0x7fb074195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.046+0000 7fb07916d700 1 -- 172.21.15.47:0/730858010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb074103440 msgr2=0x7fb07410b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.046+0000 7fb07916d700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb074103440 0x7fb07410b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.046+0000 7fb07916d700 1 -- 172.21.15.47:0/730858010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb074196380 con 0x7fb074102a90 2024-05-11T12:28:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.054+0000 7fb07916d700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074102a90 0x7fb07410ae60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c920 tx=0x7fb07000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.055+0000 7fb0667fc700 1 -- 172.21.15.47:0/730858010 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070004a30 con 0x7fb074102a90 2024-05-11T12:28:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.055+0000 7fb0667fc700 1 -- 172.21.15.47:0/730858010 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0700054d0 con 0x7fb074102a90 2024-05-11T12:28:56.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.056+0000 7fb0667fc700 1 -- 172.21.15.47:0/730858010 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070007c70 con 0x7fb074102a90 2024-05-11T12:28:56.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.057+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb074196670 con 0x7fb074102a90 2024-05-11T12:28:56.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.058+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07406bc20 con 0x7fb074102a90 2024-05-11T12:28:56.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.069+0000 7fb0667fc700 1 -- 172.21.15.47:0/730858010 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb070004c60 con 0x7fb074102a90 2024-05-11T12:28:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.078+0000 7fb085c67b80 1 Processor -- start 2024-05-11T12:28:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.078+0000 7fb085c67b80 1 -- start start 2024-05-11T12:28:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.078+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc35f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.079+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 0x55d88cc305a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.079+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc3c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.079+0000 7fb085c67b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d88cb828e0 con 0x55d88cc30ae0 2024-05-11T12:28:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.079+0000 7fb085c67b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d88cb21840 con 0x55d88cc35b50 2024-05-11T12:28:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.079+0000 7fb085c67b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d88cb7e8a0 con 0x55d88cc30120 2024-05-11T12:28:56.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.082+0000 7fb07896c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc3c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.085+0000 7fb07896c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc3c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47248/0 (socket says 172.21.15.47:47248) 2024-05-11T12:28:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.085+0000 7fb07896c700 1 -- 172.21.15.47:0/2202885841 learned_addr learned my addr 172.21.15.47:0/2202885841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.082+0000 7fb07916d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 0x55d88cc305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.083+0000 7fb07996e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc35f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.086+0000 7fb0667fc700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb0600b2a90 0x7fb0600b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.087+0000 7fb0667fc700 1 -- 172.21.15.47:0/730858010 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb0700e1d80 con 0x7fb074102a90 2024-05-11T12:28:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.088+0000 7fb07996e700 1 -- 172.21.15.47:0/2202885841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 msgr2=0x55d88cc305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.088+0000 7fb07996e700 1 --2- 172.21.15.47:0/2202885841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 0x55d88cc305a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.088+0000 7fb07996e700 1 -- 172.21.15.47:0/2202885841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 msgr2=0x55d88cc3c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.088+0000 7fb07996e700 1 --2- 172.21.15.47:0/2202885841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc3c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.088+0000 7fb07996e700 1 -- 172.21.15.47:0/2202885841 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88cacf950 con 0x55d88cc35b50 2024-05-11T12:28:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.093+0000 7fb07996e700 1 --2- 172.21.15.47:0/2202885841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc35f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb06c01a760 tx=0x7fb06c01a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=278bb3386b1863b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.093+0000 7fb07996e700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb0600b2a90 0x7fb0600b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.096+0000 7fb05f7fe700 1 -- 172.21.15.47:0/2202885841 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c003e10 con 0x55d88cc35b50 2024-05-11T12:28:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.102+0000 7fb05f7fe700 1 -- 172.21.15.47:0/2202885841 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06c035030 con 0x55d88cc35b50 2024-05-11T12:28:56.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.103+0000 7fb05f7fe700 1 -- 172.21.15.47:0/2202885841 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c02f880 con 0x55d88cc35b50 2024-05-11T12:28:56.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.106+0000 7fb085c67b80 1 -- 172.21.15.47:0/2202885841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 msgr2=0x55d88cc35f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.106+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2202885841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc35f30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb06c01a760 tx=0x7fb06c01a1a0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.109+0000 7fb07996e700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb0600b2a90 0x7fb0600b4f50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb06c038000 tx=0x7fb06c02fb00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.110+0000 7fb085c67b80 1 -- 172.21.15.47:0/2202885841 shutdown_connections 2024-05-11T12:28:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.110+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2202885841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc3c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.110+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2202885841 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc35f30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.110+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2202885841 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 0x55d88cc305a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.110+0000 7fb085c67b80 1 -- 172.21.15.47:0/2202885841 >> 172.21.15.47:0/2202885841 conn(0x55d88cb5cee0 msgr2=0x55d88cc2aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:56.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.114+0000 7fb085c67b80 1 -- 172.21.15.47:0/2202885841 shutdown_connections 2024-05-11T12:28:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.118+0000 7fb085c67b80 1 -- 172.21.15.47:0/2202885841 wait complete. 2024-05-11T12:28:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.119+0000 7fb085c67b80 1 -- start start 2024-05-11T12:28:56.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.120+0000 7fb085c67b80 1 Processor -- start 2024-05-11T12:28:56.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.122+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc3c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.122+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30ae0 0x55d88cc398e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.122+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc39e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.122+0000 7fb085c67b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55d88cc3ce50 con 0x55d88cc30120 2024-05-11T12:28:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.122+0000 7fb085c67b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55d88cae8cb0 con 0x55d88cc35b50 2024-05-11T12:28:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.122+0000 7fb085c67b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55d88cc3cfd0 con 0x55d88cc30ae0 2024-05-11T12:28:56.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.123+0000 7fb085c67b80 10 client.0 _ll_register_callbacks cb 0x55d88cb1a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:56.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.126+0000 7fb07916d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc3c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.126+0000 7fb07916d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc3c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47262/0 (socket says 172.21.15.47:47262) 2024-05-11T12:28:56.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.126+0000 7fb07916d700 1 -- 172.21.15.47:0/2296105041 learned_addr learned my addr 172.21.15.47:0/2296105041 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:56.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.126+0000 7fb07896c700 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30ae0 0x55d88cc398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.126+0000 7fb07996e700 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc39e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.130+0000 7fb07916d700 1 -- 172.21.15.47:0/2296105041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30ae0 msgr2=0x55d88cc398e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.130+0000 7fb07916d700 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30ae0 0x55d88cc398e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.130+0000 7fb07916d700 1 -- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 msgr2=0x55d88cc39e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.130+0000 7fb07916d700 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc39e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.130+0000 7fb07916d700 1 -- 172.21.15.47:0/2296105041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88cb1e710 con 0x55d88cc30120 2024-05-11T12:28:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.135+0000 7fb07916d700 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc3c4f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb0700a6560 tx=0x7fb0700a91c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade5a3dac73cec39 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:56.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.135+0000 7fb05cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:28:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.139+0000 7fb05cff9700 1 -- 172.21.15.47:0/2296105041 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07001d040 con 0x55d88cc30120 2024-05-11T12:28:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.139+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06c035030 con 0x55d88cc30120 2024-05-11T12:28:56.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.140+0000 7fb05cff9700 1 -- 172.21.15.47:0/2296105041 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb0700b1cc0 con 0x55d88cc30120 2024-05-11T12:28:56.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.146+0000 7fb05cff9700 1 -- 172.21.15.47:0/2296105041 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb0700a8e00 con 0x55d88cc30120 2024-05-11T12:28:56.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.146+0000 7fb085c67b80 10 client.15510 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:56.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.146+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d88cc474d0 con 0x55d88cc30120 2024-05-11T12:28:56.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.148+0000 7fb05cff9700 1 -- 172.21.15.47:0/2296105041 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0700a99f0 con 0x55d88cc30120 2024-05-11T12:28:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.148+0000 7fb05cff9700 1 -- 172.21.15.47:0/2296105041 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb070005000 con 0x55d88cc30120 2024-05-11T12:28:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.148+0000 7fb085c67b80 10 client.15510 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:56.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.148+0000 7fb085c67b80 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:56.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.149+0000 7fb085c67b80 20 client.15510 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:56.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.149+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88cc49f20 0x55d88cc4c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.149+0000 7fb085c67b80 4 client.15510 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:28:56.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.150+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb06c035030 con 0x55d88cc49f20 2024-05-11T12:28:56.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.152+0000 7fb07896c700 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88cc49f20 0x55d88cc4c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.156+0000 7fb07896c700 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88cc49f20 0x55d88cc4c340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29c9aeda4900bbc server_cookie=1f8d02e1c73cdd58 in_seq=0 out_seq=0 2024-05-11T12:28:56.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.156+0000 7fb05cff9700 10 client.15510 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:56.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.163+0000 7fb05cff9700 1 -- 172.21.15.47:0/2296105041 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fb06c035030 con 0x55d88cc49f20 2024-05-11T12:28:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.163+0000 7fb05cff9700 10 client.15510 handle_command_reply: tid=0 2024-05-11T12:28:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.163+0000 7fb085c67b80 1 client.15510 shutdown 2024-05-11T12:28:56.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 msgr2=0x55d88cc3c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc3c4f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fb0700a6560 tx=0x7fb0700a91c0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 shutdown_connections 2024-05-11T12:28:56.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88cc49f20 0x55d88cc4c340 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc39e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc3c4f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2296105041 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30ae0 0x55d88cc398e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.168+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 >> 172.21.15.47:0/2296105041 conn(0x55d88cb5cee0 msgr2=0x55d88cc254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:56.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.169+0000 7fb05cff9700 0 client.15510 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.171+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 shutdown_connections 2024-05-11T12:28:56.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.172+0000 7fb085c67b80 1 -- 172.21.15.47:0/2296105041 wait complete. 2024-05-11T12:28:56.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.173+0000 7fb085c67b80 20 client.15510 trim_cache size 0 max 16384 2024-05-11T12:28:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 Processor -- start 2024-05-11T12:28:56.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 -- start start 2024-05-11T12:28:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc45ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc30ae0 0x55d88cc350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc35b50 0x55d88cc24c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741a1150 con 0x55d88cc30120 2024-05-11T12:28:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741a39d0 con 0x55d88cc30ae0 2024-05-11T12:28:56.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.241+0000 7fb085c67b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741a17c0 con 0x55d88cc35b50 2024-05-11T12:28:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.242+0000 7fb07996e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc35b50 0x55d88cc24c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.242+0000 7fb07996e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc35b50 0x55d88cc24c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:34744/0 (socket says 172.21.15.47:34744) 2024-05-11T12:28:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.242+0000 7fb07996e700 1 -- 172.21.15.47:0/2692090663 learned_addr learned my addr 172.21.15.47:0/2692090663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:56.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.242+0000 7fb07896c700 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc30ae0 0x55d88cc350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.243+0000 7fb07996e700 1 -- 172.21.15.47:0/2692090663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc30ae0 msgr2=0x55d88cc350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.243+0000 7fb07996e700 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc30ae0 0x55d88cc350d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.243+0000 7fb07996e700 1 -- 172.21.15.47:0/2692090663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 msgr2=0x55d88cc45ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:28:56.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.243+0000 7fb07996e700 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc45ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.243+0000 7fb07996e700 1 -- 172.21.15.47:0/2692090663 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88cc3c860 con 0x55d88cc35b50 2024-05-11T12:28:56.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.244+0000 7fb07996e700 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc35b50 0x55d88cc24c50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb06c01c420 tx=0x7fb06c03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c4a93fe5771dfa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.245+0000 7fb05effd700 1 -- 172.21.15.47:0/2692090663 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c01acb0 con 0x55d88cc35b50 2024-05-11T12:28:56.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.245+0000 7fb05effd700 1 -- 172.21.15.47:0/2692090663 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06c035070 con 0x55d88cc35b50 2024-05-11T12:28:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.245+0000 7fb05effd700 1 -- 172.21.15.47:0/2692090663 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0184f0 con 0x55d88cc35b50 2024-05-11T12:28:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.247+0000 7fb085c67b80 1 -- 172.21.15.47:0/2692090663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc35b50 msgr2=0x55d88cc24c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.247+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc35b50 0x55d88cc24c50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb06c01c420 tx=0x7fb06c03f000 comp rx=0 tx=0).stop 2024-05-11T12:28:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.249+0000 7fb085c67b80 1 -- 172.21.15.47:0/2692090663 shutdown_connections 2024-05-11T12:28:56.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.249+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc35b50 0x55d88cc24c50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.249+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30120 0x55d88cc45ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.249+0000 7fb085c67b80 1 --2- 172.21.15.47:0/2692090663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc30ae0 0x55d88cc350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.249+0000 7fb085c67b80 1 -- 172.21.15.47:0/2692090663 >> 172.21.15.47:0/2692090663 conn(0x55d88cb5cee0 msgr2=0x55d88cc2f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:56.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.251+0000 7fb085c67b80 1 -- 172.21.15.47:0/2692090663 shutdown_connections 2024-05-11T12:28:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.251+0000 7fb085c67b80 1 -- 172.21.15.47:0/2692090663 wait complete. 2024-05-11T12:28:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.251+0000 7fb085c67b80 1 -- start start 2024-05-11T12:28:56.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.251+0000 7fb085c67b80 1 Processor -- start 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 0x55d88cc42400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc42940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb085c67b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb085c67b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741a1e90 con 0x55d88cc30ae0 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb085c67b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741a3300 con 0x55d88cc35b50 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb085c67b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741a2560 con 0x55d88cc30120 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb085c67b80 10 client.0 _ll_register_callbacks cb 0x55d88cb1a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb07896c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc42940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb07896c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc42940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47280/0 (socket says 172.21.15.47:47280) 2024-05-11T12:28:56.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.254+0000 7fb07896c700 1 -- 172.21.15.47:0/3757050656 learned_addr learned my addr 172.21.15.47:0/3757050656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:28:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.255+0000 7fb07896c700 1 -- 172.21.15.47:0/3757050656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 msgr2=0x55d88cc42400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:28:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.255+0000 7fb07896c700 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 0x55d88cc42400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.255+0000 7fb07896c700 1 -- 172.21.15.47:0/3757050656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 msgr2=0x55d88cc3f250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:28:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.255+0000 7fb07896c700 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc3f250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.255+0000 7fb07896c700 1 -- 172.21.15.47:0/3757050656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88cc3fa90 con 0x55d88cc30ae0 2024-05-11T12:28:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.255+0000 7fb07896c700 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc42940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb068015010 tx=0x7fb068009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c63ca3e8593f98 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:28:56.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.255+0000 7fb05cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:28:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.256+0000 7fb05cff9700 1 -- 172.21.15.47:0/3757050656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068020860 con 0x55d88cc30ae0 2024-05-11T12:28:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.256+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06c035070 con 0x55d88cc30ae0 2024-05-11T12:28:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.256+0000 7fb05cff9700 1 -- 172.21.15.47:0/3757050656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fb068020a00 con 0x55d88cc30ae0 2024-05-11T12:28:56.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.257+0000 7fb05cff9700 1 -- 172.21.15.47:0/3757050656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb068020c00 con 0x55d88cc30ae0 2024-05-11T12:28:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.259+0000 7fb085c67b80 10 client.15513 fetch_fsmap learned FSMap version 20 2024-05-11T12:28:56.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.259+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d88ccb5490 con 0x55d88cc30ae0 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb05cff9700 1 -- 172.21.15.47:0/3757050656 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb06c035070 con 0x55d88cc30ae0 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb05cff9700 1 -- 172.21.15.47:0/3757050656 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fb06800e040 con 0x55d88cc30ae0 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb085c67b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb085c67b80 10 client.15513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb085c67b80 20 client.15513 populate_metadata read hostname 'smithi047' 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb085c67b80 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88ccb5bd0 0x55d88ccb7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb085c67b80 4 client.15513 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:28:56.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.260+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d88cb902c0 con 0x55d88ccb5bd0 2024-05-11T12:28:56.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.264+0000 7fb07916d700 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88ccb5bd0 0x55d88ccb7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:28:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.269+0000 7fb05cff9700 10 client.15513 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:56.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.269+0000 7fb07916d700 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88ccb5bd0 0x55d88ccb7ff0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8afd4ebf4c86da52 server_cookie=dad50b3d4659293c in_seq=0 out_seq=0 2024-05-11T12:28:56.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.585+0000 7fb05cff9700 1 -- 172.21.15.47:0/3757050656 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d88cb902c0 con 0x55d88ccb5bd0 2024-05-11T12:28:56.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.585+0000 7fb05cff9700 10 client.15513 handle_command_reply: tid=0 2024-05-11T12:28:56.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.585+0000 7fb085c67b80 1 client.15513 shutdown 2024-05-11T12:28:56.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 msgr2=0x55d88cc42940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc42940 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb068015010 tx=0x7fb068009b90 comp rx=0 tx=0).stop 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 shutdown_connections 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55d88ccb5bd0 0x55d88ccb7ff0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55d88cc35b50 0x55d88cc3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55d88cc30120 0x55d88cc42400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 --2- 172.21.15.47:0/3757050656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55d88cc30ae0 0x55d88cc42940 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.586+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 >> 172.21.15.47:0/3757050656 conn(0x55d88cb5cee0 msgr2=0x55d88cc2f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.587+0000 7fb05cff9700 0 client.15513 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.587+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 shutdown_connections 2024-05-11T12:28:56.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.587+0000 7fb085c67b80 1 -- 172.21.15.47:0/3757050656 wait complete. 2024-05-11T12:28:56.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.588+0000 7fb085c67b80 20 client.15513 trim_cache size 0 max 16384 2024-05-11T12:28:56.590 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:28:56.590 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:28:56.590 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "e0ba420d-027d-4d14-9f0a-88ab7f754484", 2024-05-11T12:28:56.590 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:28:56.590 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:28:56.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.612+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb0600b2a90 msgr2=0x7fb0600b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.612+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb0600b2a90 0x7fb0600b4f50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb06c038000 tx=0x7fb06c02fb00 comp rx=0 tx=0).stop 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.612+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074102a90 msgr2=0x7fb07410ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.612+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074102a90 0x7fb07410ae60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb07000c920 tx=0x7fb07000cc30 comp rx=0 tx=0).stop 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.613+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 shutdown_connections 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.613+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fb0741077e0 0x7fb074195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.613+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb074102a90 0x7fb07410ae60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.613+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fb0600b2a90 0x7fb0600b4f50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.613+0000 7fb07b3d1700 1 --2- 172.21.15.47:0/730858010 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fb074103440 0x7fb07410b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:28:56.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.613+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 >> 172.21.15.47:0/730858010 conn(0x7fb074079400 msgr2=0x7fb0741006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:28:56.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.615+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 shutdown_connections 2024-05-11T12:28:56.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:28:56.615+0000 7fb07b3d1700 1 -- 172.21.15.47:0/730858010 wait complete. 2024-05-11T12:28:57.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:57 smithi050 ceph-mon[34072]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 MiB/s wr, 1.66k op/s 2024-05-11T12:28:57.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:57 smithi047 ceph-mon[30375]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 MiB/s wr, 1.66k op/s 2024-05-11T12:28:57.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:57 smithi064 ceph-mon[34537]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 783 MiB/s wr, 1.66k op/s 2024-05-11T12:28:58.169 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:58 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:28:58.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:58 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:58.170 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:58 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:28:58.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:58 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:28:58.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:58 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:58.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:58 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:28:58.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:58 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:28:58.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:58 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:58.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:58 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:28:58.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:28:59.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:59 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:59.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:59 smithi047 ceph-mon[30375]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 879 MiB/s wr, 1.66k op/s 2024-05-11T12:28:59.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:28:59 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:28:59.565 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:28:59.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:59 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:59.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:59 smithi064 ceph-mon[34537]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 879 MiB/s wr, 1.66k op/s 2024-05-11T12:28:59.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:28:59 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:28:59.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:59 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:28:59.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:59 smithi050 ceph-mon[34072]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 879 MiB/s wr, 1.66k op/s 2024-05-11T12:28:59.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:28:59 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:29:01.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:01 smithi047 ceph-mon[30375]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 743 MiB/s wr, 1.33k op/s 2024-05-11T12:29:01.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:01 smithi064 ceph-mon[34537]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 743 MiB/s wr, 1.33k op/s 2024-05-11T12:29:01.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:01 smithi050 ceph-mon[34072]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 743 MiB/s wr, 1.33k op/s 2024-05-11T12:29:03.228 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:03 smithi047 ceph-mon[30375]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-05-11T12:29:03.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:03 smithi064 ceph-mon[34537]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-05-11T12:29:03.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:03 smithi050 ceph-mon[34072]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-05-11T12:29:05.521 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:05 smithi047 ceph-mon[30375]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 788 MiB/s wr, 1.51k op/s 2024-05-11T12:29:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.520+0000 7febc41e5700 1 -- 172.21.15.47:0/3893105522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc10aa90 msgr2=0x7febbc10af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.520+0000 7febc41e5700 1 --2- 172.21.15.47:0/3893105522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc10aa90 0x7febbc10af10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7febb8009990 tx=0x7febb8009ca0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.524+0000 7febc41e5700 1 -- 172.21.15.47:0/3893105522 shutdown_connections 2024-05-11T12:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.524+0000 7febc41e5700 1 --2- 172.21.15.47:0/3893105522 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febbc11cd30 0x7febbc11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.524+0000 7febc41e5700 1 --2- 172.21.15.47:0/3893105522 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febbc111d30 0x7febbc10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.524+0000 7febc41e5700 1 --2- 172.21.15.47:0/3893105522 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc10aa90 0x7febbc10af10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.524+0000 7febc41e5700 1 -- 172.21.15.47:0/3893105522 >> 172.21.15.47:0/3893105522 conn(0x7febbc06e6c0 msgr2=0x7febbc06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:05.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.524+0000 7febc41e5700 1 -- 172.21.15.47:0/3893105522 shutdown_connections 2024-05-11T12:29:05.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.525+0000 7febc41e5700 1 -- 172.21.15.47:0/3893105522 wait complete. 2024-05-11T12:29:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.526+0000 7febc41e5700 1 Processor -- start 2024-05-11T12:29:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.526+0000 7febc41e5700 1 -- start start 2024-05-11T12:29:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc41e5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febbc10aa90 0x7febbc1ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc41e5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febbc111d30 0x7febbc1aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc41e5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc11cd30 0x7febbc1a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc41e5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7febbc122750 con 0x7febbc11cd30 2024-05-11T12:29:05.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc41e5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7febbc1225d0 con 0x7febbc10aa90 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc41e5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febbc1228d0 con 0x7febbc111d30 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc2782700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc11cd30 0x7febbc1a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc2782700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc11cd30 0x7febbc1a89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57628/0 (socket says 172.21.15.47:57628) 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc2782700 1 -- 172.21.15.47:0/2625962662 learned_addr learned my addr 172.21.15.47:0/2625962662 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.527+0000 7febc1780700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febbc111d30 0x7febbc1aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.528+0000 7febc2782700 1 -- 172.21.15.47:0/2625962662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febbc111d30 msgr2=0x7febbc1aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.528+0000 7febc2782700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febbc111d30 0x7febbc1aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.528+0000 7febc2782700 1 -- 172.21.15.47:0/2625962662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febbc10aa90 msgr2=0x7febbc1ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.529+0000 7febc2782700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febbc10aa90 0x7febbc1ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.529+0000 7febc2782700 1 -- 172.21.15.47:0/2625962662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc1a9110 con 0x7febbc11cd30 2024-05-11T12:29:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.529+0000 7febc2782700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc11cd30 0x7febbc1a89b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7febb400b7b0 tx=0x7febb400bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.530+0000 7febb2ffd700 1 -- 172.21.15.47:0/2625962662 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4014070 con 0x7febbc11cd30 2024-05-11T12:29:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.530+0000 7febb2ffd700 1 -- 172.21.15.47:0/2625962662 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb4010cb0 con 0x7febbc11cd30 2024-05-11T12:29:05.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.530+0000 7febb2ffd700 1 -- 172.21.15.47:0/2625962662 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4017d30 con 0x7febbc11cd30 2024-05-11T12:29:05.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.530+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc1a9400 con 0x7febbc11cd30 2024-05-11T12:29:05.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.530+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc112510 con 0x7febbc11cd30 2024-05-11T12:29:05.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.535+0000 7febb2ffd700 1 -- 172.21.15.47:0/2625962662 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7febb4017070 con 0x7febbc11cd30 2024-05-11T12:29:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.537+0000 7febb2ffd700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feba80b2a10 0x7feba80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.538+0000 7febb2ffd700 1 -- 172.21.15.47:0/2625962662 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7febb40e1af0 con 0x7febbc11cd30 2024-05-11T12:29:05.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.539+0000 7febc1f81700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feba80b2a10 0x7feba80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.540+0000 7febc1f81700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feba80b2a10 0x7feba80b4ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7febbc1a9af0 tx=0x7febac005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:05.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.541+0000 7febcea7bb80 1 Processor -- start 2024-05-11T12:29:05.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.541+0000 7febcea7bb80 1 -- start start 2024-05-11T12:29:05.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.542+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.542+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb666a0 0x55782fb61510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.542+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb61b30 0x55782fb61f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.542+0000 7febcea7bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55782fab3940 con 0x55782fb61b30 2024-05-11T12:29:05.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.542+0000 7febcea7bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55782fa18da0 con 0x55782fb65d80 2024-05-11T12:29:05.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.542+0000 7febcea7bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55782faaf900 con 0x55782fb666a0 2024-05-11T12:29:05.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.543+0000 7febc1f81700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.543+0000 7febc1f81700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb66160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:32892/0 (socket says 172.21.15.47:32892) 2024-05-11T12:29:05.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.543+0000 7febc1f81700 1 -- 172.21.15.47:0/3102783629 learned_addr learned my addr 172.21.15.47:0/3102783629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.543+0000 7febc2782700 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb61b30 0x55782fb61f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.543+0000 7febc2782700 1 -- 172.21.15.47:0/3102783629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb666a0 msgr2=0x55782fb61510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:29:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.543+0000 7febc2782700 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb666a0 0x55782fb61510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.543+0000 7febc2782700 1 -- 172.21.15.47:0/3102783629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 msgr2=0x55782fb66160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.544+0000 7febc2782700 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb66160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.544+0000 7febc2782700 1 -- 172.21.15.47:0/3102783629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55782fa008d0 con 0x55782fb61b30 2024-05-11T12:29:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.544+0000 7febc2782700 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb61b30 0x55782fb61f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7febb400b780 tx=0x7febb40a8b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382b8b5304e27be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:05.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.545+0000 7feba7fff700 1 -- 172.21.15.47:0/3102783629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb401e070 con 0x55782fb61b30 2024-05-11T12:29:05.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.545+0000 7feba7fff700 1 -- 172.21.15.47:0/3102783629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb40ad480 con 0x55782fb61b30 2024-05-11T12:29:05.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.545+0000 7feba7fff700 1 -- 172.21.15.47:0/3102783629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb40b1a10 con 0x55782fb61b30 2024-05-11T12:29:05.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.546+0000 7febcea7bb80 1 -- 172.21.15.47:0/3102783629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb61b30 msgr2=0x55782fb61f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.546+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb61b30 0x55782fb61f90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7febb400b780 tx=0x7febb40a8b30 comp rx=0 tx=0).stop 2024-05-11T12:29:05.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.547+0000 7febcea7bb80 1 -- 172.21.15.47:0/3102783629 shutdown_connections 2024-05-11T12:29:05.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.547+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb61b30 0x55782fb61f90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.547+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb66160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.547+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3102783629 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb666a0 0x55782fb61510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.547+0000 7febcea7bb80 1 -- 172.21.15.47:0/3102783629 >> 172.21.15.47:0/3102783629 conn(0x55782fa8db70 msgr2=0x55782fb5bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:05.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.547+0000 7febcea7bb80 1 -- 172.21.15.47:0/3102783629 shutdown_connections 2024-05-11T12:29:05.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.547+0000 7febcea7bb80 1 -- 172.21.15.47:0/3102783629 wait complete. 2024-05-11T12:29:05.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.548+0000 7febcea7bb80 1 -- start start 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.548+0000 7febcea7bb80 1 Processor -- start 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.549+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb61b30 0x55782fb6d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.549+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb65d80 0x55782fb6a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.549+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.549+0000 7febcea7bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55782fb6e210 con 0x55782fb666a0 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.549+0000 7febcea7bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55782fa6ced0 con 0x55782fb61b30 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.549+0000 7febcea7bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55782fb6e390 con 0x55782fb65d80 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.549+0000 7febcea7bb80 10 client.0 _ll_register_callbacks cb 0x55782fa4b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.550+0000 7febc1780700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb65d80 0x55782fb6a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.550+0000 7febc1780700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb65d80 0x55782fb6a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:33400/0 (socket says 172.21.15.47:33400) 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.550+0000 7febc1780700 1 -- 172.21.15.47:0/2916153121 learned_addr learned my addr 172.21.15.47:0/2916153121 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.550+0000 7febc1f81700 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb61b30 0x55782fb6d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.551+0000 7febc1780700 1 -- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb61b30 msgr2=0x55782fb6d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:05.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.551+0000 7febc1780700 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb61b30 0x55782fb6d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.551+0000 7febc1780700 1 -- 172.21.15.47:0/2916153121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 msgr2=0x55782fb6ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:05.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.551+0000 7febc1780700 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.551+0000 7febc1780700 1 -- 172.21.15.47:0/2916153121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55782fa6a760 con 0x55782fb65d80 2024-05-11T12:29:05.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.553+0000 7febc1780700 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb65d80 0x55782fb6a920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febb8015000 tx=0x7febb8005c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5705c6061c01af98 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:05.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.553+0000 7feba57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:29:05.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.553+0000 7feba57fa700 1 -- 172.21.15.47:0/2916153121 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8028060 con 0x55782fb65d80 2024-05-11T12:29:05.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.553+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb40ad480 con 0x55782fb65d80 2024-05-11T12:29:05.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.553+0000 7feba57fa700 1 -- 172.21.15.47:0/2916153121 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7febb8003c90 con 0x55782fb65d80 2024-05-11T12:29:05.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.554+0000 7feba57fa700 1 -- 172.21.15.47:0/2916153121 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7febb8003680 con 0x55782fb65d80 2024-05-11T12:29:05.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.554+0000 7febcea7bb80 10 client.25153 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:05.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.554+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55782fb78b30 con 0x55782fb65d80 2024-05-11T12:29:05.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.557+0000 7feba57fa700 1 -- 172.21.15.47:0/2916153121 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb40ad480 con 0x55782fb65d80 2024-05-11T12:29:05.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.558+0000 7feba57fa700 1 -- 172.21.15.47:0/2916153121 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7febb800f7b0 con 0x55782fb65d80 2024-05-11T12:29:05.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.566+0000 7febcea7bb80 10 client.25153 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:05.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.566+0000 7febcea7bb80 10 client.25153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:05.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.566+0000 7febcea7bb80 20 client.25153 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:05.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.566+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fb7b0c0 0x55782fb7d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.566+0000 7febcea7bb80 4 client.25153 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:29:05.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.566+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55782fac1320 con 0x55782fb7b0c0 2024-05-11T12:29:05.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.568+0000 7febc1f81700 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fb7b0c0 0x55782fb7d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.573+0000 7febc1f81700 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fb7b0c0 0x55782fb7d510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e7801ab7792ee0 server_cookie=718cee86f0cdc4be in_seq=0 out_seq=0 2024-05-11T12:29:05.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.574+0000 7feba57fa700 10 client.25153 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:05.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.577+0000 7feba57fa700 1 -- 172.21.15.47:0/2916153121 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55782fac1320 con 0x55782fb7b0c0 2024-05-11T12:29:05.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.577+0000 7feba57fa700 10 client.25153 handle_command_reply: tid=0 2024-05-11T12:29:05.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.577+0000 7febcea7bb80 1 client.25153 shutdown 2024-05-11T12:29:05.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.581+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb65d80 msgr2=0x55782fb6a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.581+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb65d80 0x55782fb6a920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7febb8015000 tx=0x7febb8005c20 comp rx=0 tx=0).stop 2024-05-11T12:29:05.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.583+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 shutdown_connections 2024-05-11T12:29:05.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.583+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fb7b0c0 0x55782fb7d510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.583+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb6ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.583+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb61b30 0x55782fb6d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.583+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2916153121 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb65d80 0x55782fb6a920 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.583+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 >> 172.21.15.47:0/2916153121 conn(0x55782fa8db70 msgr2=0x55782fb56830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:05.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.585+0000 7feba57fa700 0 client.25153 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:05.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.585+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 shutdown_connections 2024-05-11T12:29:05.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.585+0000 7febcea7bb80 1 -- 172.21.15.47:0/2916153121 wait complete. 2024-05-11T12:29:05.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.585+0000 7febcea7bb80 20 client.25153 trim_cache size 0 max 16384 2024-05-11T12:29:05.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:05 smithi064 ceph-mon[34537]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 788 MiB/s wr, 1.51k op/s 2024-05-11T12:29:05.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.649+0000 7febcea7bb80 1 Processor -- start 2024-05-11T12:29:05.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.651+0000 7febcea7bb80 1 -- start start 2024-05-11T12:29:05.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.652+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb772a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.653+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb56940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.653+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.653+0000 7febcea7bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7febbc117cd0 con 0x55782fb666a0 2024-05-11T12:29:05.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.654+0000 7febcea7bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7febbc11a550 con 0x55782fb65d80 2024-05-11T12:29:05.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.654+0000 7febcea7bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febbc118340 con 0x55782fb61b30 2024-05-11T12:29:05.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.658+0000 7febc1780700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb56940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.659+0000 7febc1780700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb56940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:32910/0 (socket says 172.21.15.47:32910) 2024-05-11T12:29:05.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.659+0000 7febc1780700 1 -- 172.21.15.47:0/3164470838 learned_addr learned my addr 172.21.15.47:0/3164470838 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:05.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.660+0000 7febc1f81700 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb772a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.661+0000 7febc1780700 1 -- 172.21.15.47:0/3164470838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 msgr2=0x55782fb772a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:05.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.661+0000 7febc1780700 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb772a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.661+0000 7febc1780700 1 -- 172.21.15.47:0/3164470838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 msgr2=0x55782fb682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:05.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.661+0000 7febc1780700 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.661+0000 7febc1780700 1 -- 172.21.15.47:0/3164470838 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55782fb58980 con 0x55782fb65d80 2024-05-11T12:29:05.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.664+0000 7febc1780700 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb56940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7febb8000c00 tx=0x7febb800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6372936c615c6e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:05.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.669+0000 7feba77fe700 1 -- 172.21.15.47:0/3164470838 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8028030 con 0x55782fb65d80 2024-05-11T12:29:05.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.669+0000 7feba77fe700 1 -- 172.21.15.47:0/3164470838 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb80173f0 con 0x55782fb65d80 2024-05-11T12:29:05.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.669+0000 7feba77fe700 1 -- 172.21.15.47:0/3164470838 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8005490 con 0x55782fb65d80 2024-05-11T12:29:05.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.676+0000 7febcea7bb80 1 -- 172.21.15.47:0/3164470838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 msgr2=0x55782fb56940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.676+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb56940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7febb8000c00 tx=0x7febb800bcf0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.683+0000 7febcea7bb80 1 -- 172.21.15.47:0/3164470838 shutdown_connections 2024-05-11T12:29:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.683+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.683+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb772a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.683+0000 7febcea7bb80 1 --2- 172.21.15.47:0/3164470838 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb56940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.683+0000 7febcea7bb80 1 -- 172.21.15.47:0/3164470838 >> 172.21.15.47:0/3164470838 conn(0x55782fa8db70 msgr2=0x55782fb67bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.684+0000 7febcea7bb80 1 -- 172.21.15.47:0/3164470838 shutdown_connections 2024-05-11T12:29:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.684+0000 7febcea7bb80 1 -- 172.21.15.47:0/3164470838 wait complete. 2024-05-11T12:29:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.685+0000 7febcea7bb80 1 -- start start 2024-05-11T12:29:05.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.686+0000 7febcea7bb80 1 Processor -- start 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb73620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febcea7bb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febcea7bb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7febbc118a10 con 0x55782fb666a0 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febcea7bb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7febbc119e80 con 0x55782fb65d80 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febc2782700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febc2782700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb829b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57656/0 (socket says 172.21.15.47:57656) 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febc2782700 1 -- 172.21.15.47:0/2084120396 learned_addr learned my addr 172.21.15.47:0/2084120396 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.693+0000 7febcea7bb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7febbc1190e0 con 0x55782fb61b30 2024-05-11T12:29:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.694+0000 7febcea7bb80 10 client.0 _ll_register_callbacks cb 0x55782fa4b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:05.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:05 smithi050 ceph-mon[34072]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 788 MiB/s wr, 1.51k op/s 2024-05-11T12:29:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.713+0000 7febc1f81700 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb73620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.713+0000 7febc1780700 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.719+0000 7febc1780700 1 -- 172.21.15.47:0/2084120396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 msgr2=0x55782fb73620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.719+0000 7febc1780700 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb73620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.719+0000 7febc1780700 1 -- 172.21.15.47:0/2084120396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 msgr2=0x55782fb829b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.719+0000 7febc1780700 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb829b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.719+0000 7febc1780700 1 -- 172.21.15.47:0/2084120396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55782fb82ef0 con 0x55782fb65d80 2024-05-11T12:29:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.728+0000 7febc1780700 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb73b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7febb80098e0 tx=0x7febb800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2868115f6ef990da server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.728+0000 7feba57fa700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:29:05.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.732+0000 7feba57fa700 1 -- 172.21.15.47:0/2084120396 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb8005310 con 0x55782fb65d80 2024-05-11T12:29:05.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.732+0000 7feba57fa700 1 -- 172.21.15.47:0/2084120396 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7febb80054b0 con 0x55782fb65d80 2024-05-11T12:29:05.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.733+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febb80173f0 con 0x55782fb65d80 2024-05-11T12:29:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.735+0000 7feba57fa700 1 -- 172.21.15.47:0/2084120396 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7febb8044050 con 0x55782fb65d80 2024-05-11T12:29:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.736+0000 7febcea7bb80 10 client.25121 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:05.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.736+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55782fbe60b0 con 0x55782fb65d80 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.741+0000 7feba57fa700 1 -- 172.21.15.47:0/2084120396 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb803b070 con 0x55782fb65d80 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.741+0000 7feba57fa700 1 -- 172.21.15.47:0/2084120396 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7febb8035070 con 0x55782fb65d80 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.741+0000 7febcea7bb80 10 client.25121 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.741+0000 7febcea7bb80 10 client.25121 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.742+0000 7febcea7bb80 20 client.25121 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.742+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fbe6630 0x55782fbe8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.742+0000 7febcea7bb80 4 client.25121 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:29:05.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.742+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7febb80173f0 con 0x55782fbe6630 2024-05-11T12:29:05.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.746+0000 7febc1f81700 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fbe6630 0x55782fbe8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:05.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.750+0000 7febc1f81700 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fbe6630 0x55782fbe8a50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864eba901c3bf2ef server_cookie=9ad1b84391ec385c in_seq=0 out_seq=0 2024-05-11T12:29:05.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.754+0000 7feba57fa700 10 client.25121 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:05.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.754+0000 7feba57fa700 1 -- 172.21.15.47:0/2084120396 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7febb80173f0 con 0x55782fbe6630 2024-05-11T12:29:05.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.755+0000 7feba57fa700 10 client.25121 handle_command_reply: tid=0 2024-05-11T12:29:05.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.755+0000 7febcea7bb80 1 client.25121 shutdown 2024-05-11T12:29:05.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.756+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 msgr2=0x55782fb73b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.756+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb73b60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7febb80098e0 tx=0x7febb800f690 comp rx=0 tx=0).stop 2024-05-11T12:29:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.756+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 shutdown_connections 2024-05-11T12:29:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.757+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55782fbe6630 0x55782fbe8a50 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.757+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55782fb666a0 0x55782fb829b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.757+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55782fb61b30 0x55782fb73620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.757+0000 7febcea7bb80 1 --2- 172.21.15.47:0/2084120396 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55782fb65d80 0x55782fb73b60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.757+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 >> 172.21.15.47:0/2084120396 conn(0x55782fa8db70 msgr2=0x55782fb69aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:05.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.757+0000 7feba57fa700 0 client.25121 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:05.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.767+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 shutdown_connections 2024-05-11T12:29:05.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.768+0000 7febcea7bb80 1 -- 172.21.15.47:0/2084120396 wait complete. 2024-05-11T12:29:05.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.768+0000 7febcea7bb80 20 client.25121 trim_cache size 0 max 16384 2024-05-11T12:29:05.769 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:29:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:29:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:29:05.770 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:29:05.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feba80b2a10 msgr2=0x7feba80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feba80b2a10 0x7feba80b4ed0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7febbc1a9af0 tx=0x7febac005c80 comp rx=0 tx=0).stop 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc11cd30 msgr2=0x7febbc1a89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc11cd30 0x7febbc1a89b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7febb400b7b0 tx=0x7febb400bac0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 shutdown_connections 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7febbc11cd30 0x7febbc1a89b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7feba80b2a10 0x7feba80b4ed0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7febbc10aa90 0x7febbc1ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 --2- 172.21.15.47:0/2625962662 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7febbc111d30 0x7febbc1aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:05.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.770+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 >> 172.21.15.47:0/2625962662 conn(0x7febbc06e6c0 msgr2=0x7febbc10f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:05.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.772+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 shutdown_connections 2024-05-11T12:29:05.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:05.772+0000 7febc41e5700 1 -- 172.21.15.47:0/2625962662 wait complete. 2024-05-11T12:29:07.536 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:07 smithi047 ceph-mon[30375]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 825 MiB/s wr, 1.72k op/s 2024-05-11T12:29:07.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:07 smithi064 ceph-mon[34537]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 825 MiB/s wr, 1.72k op/s 2024-05-11T12:29:07.659 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:29:07.659 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:29:07.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:07 smithi050 ceph-mon[34072]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 825 MiB/s wr, 1.72k op/s 2024-05-11T12:29:08.785 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:29:09.274 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:09 smithi047 ceph-mon[30375]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 897 MiB/s wr, 1.85k op/s 2024-05-11T12:29:09.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:09 smithi064 ceph-mon[34537]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 897 MiB/s wr, 1.85k op/s 2024-05-11T12:29:09.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:09 smithi050 ceph-mon[34072]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 897 MiB/s wr, 1.85k op/s 2024-05-11T12:29:11.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:11 smithi064 ceph-mon[34537]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 768 MiB/s wr, 1.65k op/s 2024-05-11T12:29:11.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:11 smithi050 ceph-mon[34072]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 768 MiB/s wr, 1.65k op/s 2024-05-11T12:29:11.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:11 smithi047 ceph-mon[30375]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 768 MiB/s wr, 1.65k op/s 2024-05-11T12:29:13.463 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:13 smithi047 ceph-mon[30375]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 867 MiB/s wr, 1.85k op/s 2024-05-11T12:29:13.613 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:13 smithi064 ceph-mon[34537]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 867 MiB/s wr, 1.85k op/s 2024-05-11T12:29:13.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:13 smithi050 ceph-mon[34072]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 867 MiB/s wr, 1.85k op/s 2024-05-11T12:29:14.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.295+0000 7f3641e41700 1 -- 172.21.15.47:0/3654008414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 msgr2=0x7f363c10ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:14.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.298+0000 7f3641e41700 1 --2- 172.21.15.47:0/3654008414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10ec00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f363400cac0 tx=0x7f363400cdd0 comp rx=0 tx=0).stop 2024-05-11T12:29:14.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.302+0000 7f3641e41700 1 -- 172.21.15.47:0/3654008414 shutdown_connections 2024-05-11T12:29:14.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.302+0000 7f3641e41700 1 --2- 172.21.15.47:0/3654008414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363c10f2e0 0x7f363c116eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:14.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.302+0000 7f3641e41700 1 --2- 172.21.15.47:0/3654008414 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f363c10a7b0 0x7f363c10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:14.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.302+0000 7f3641e41700 1 --2- 172.21.15.47:0/3654008414 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10ec00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:14.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.302+0000 7f3641e41700 1 -- 172.21.15.47:0/3654008414 >> 172.21.15.47:0/3654008414 conn(0x7f363c06e6c0 msgr2=0x7f363c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.309+0000 7f3641e41700 1 -- 172.21.15.47:0/3654008414 shutdown_connections 2024-05-11T12:29:14.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.309+0000 7f3641e41700 1 -- 172.21.15.47:0/3654008414 wait complete. 2024-05-11T12:29:14.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.311+0000 7f3641e41700 1 Processor -- start 2024-05-11T12:29:14.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.312+0000 7f3641e41700 1 -- start start 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.312+0000 7f3641e41700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363c10a7b0 0x7f363c10bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f3641e41700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f3641e41700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f363c10f2e0 0x7f363c10c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f3641e41700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f363c119ea0 con 0x7f363c10a7b0 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f3641e41700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f363c119d20 con 0x7f363c10f2e0 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f3641e41700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f363c11a020 con 0x7f363c10b160 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f363affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:14.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f363affd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:57648/0 (socket says 172.21.15.47:57648) 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f363affd700 1 -- 172.21.15.47:0/1400532622 learned_addr learned my addr 172.21.15.47:0/1400532622 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.313+0000 7f363b7fe700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363c10a7b0 0x7f363c10bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.314+0000 7f363affd700 1 -- 172.21.15.47:0/1400532622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f363c10f2e0 msgr2=0x7f363c10c720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.314+0000 7f363affd700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f363c10f2e0 0x7f363c10c720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.314+0000 7f363affd700 1 -- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363c10a7b0 msgr2=0x7f363c10bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.314+0000 7f363affd700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363c10a7b0 0x7f363c10bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.314+0000 7f363affd700 1 -- 172.21.15.47:0/1400532622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1b4c00 con 0x7f363c10b160 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.315+0000 7f363affd700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10c1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36340095e0 tx=0x7f363400ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.316+0000 7f3638ff9700 1 -- 172.21.15.47:0/1400532622 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634023070 con 0x7f363c10b160 2024-05-11T12:29:14.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.316+0000 7f3638ff9700 1 -- 172.21.15.47:0/1400532622 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f363400bd00 con 0x7f363c10b160 2024-05-11T12:29:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.317+0000 7f3638ff9700 1 -- 172.21.15.47:0/1400532622 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634007c70 con 0x7f363c10b160 2024-05-11T12:29:14.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.318+0000 7f3641e41700 1 -- 172.21.15.47:0/1400532622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1b4e30 con 0x7f363c10b160 2024-05-11T12:29:14.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.318+0000 7f3641e41700 1 -- 172.21.15.47:0/1400532622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1b51d0 con 0x7f363c10b160 2024-05-11T12:29:14.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.325+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363c10b770 con 0x7f363c10b160 2024-05-11T12:29:14.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.332+0000 7f3638ff9700 1 -- 172.21.15.47:0/1400532622 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f363400b260 con 0x7f363c10b160 2024-05-11T12:29:14.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.333+0000 7f3638ff9700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36240b28f0 0x7f36240b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.335+0000 7f3638ff9700 1 -- 172.21.15.47:0/1400532622 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f36340e2c70 con 0x7f363c10b160 2024-05-11T12:29:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.335+0000 7f3638ff9700 1 -- 172.21.15.47:0/1400532622 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36340aa8f0 con 0x7f363c10b160 2024-05-11T12:29:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.336+0000 7f363b7fe700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36240b28f0 0x7f36240b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:14.337+0000 7f363b7fe700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36240b28f0 0x7f36240b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3630006fd0 tx=0x7f3630009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:15.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.051+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f363c10d1d0 con 0x7f363c10b160 2024-05-11T12:29:15.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.054+0000 7f3638ff9700 1 -- 172.21.15.47:0/1400532622 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f36340aa1f0 con 0x7f363c10b160 2024-05-11T12:29:15.056 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:29:15.056 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:29:15.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36240b28f0 msgr2=0x7f36240b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:15.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36240b28f0 0x7f36240b4db0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3630006fd0 tx=0x7f3630009190 comp rx=0 tx=0).stop 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 msgr2=0x7f363c10c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10c1e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f36340095e0 tx=0x7f363400ef70 comp rx=0 tx=0).stop 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 shutdown_connections 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f36240b28f0 0x7f36240b4db0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f363c10f2e0 0x7f363c10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f363c10a7b0 0x7f363c10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 --2- 172.21.15.47:0/1400532622 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f363c10b160 0x7f363c10c1e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:15.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.065+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 >> 172.21.15.47:0/1400532622 conn(0x7f363c06e6c0 msgr2=0x7f363c10dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:15.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.067+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 shutdown_connections 2024-05-11T12:29:15.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:15.068+0000 7f36227fc700 1 -- 172.21.15.47:0/1400532622 wait complete. 2024-05-11T12:29:15.071 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:29:15.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:15 smithi064 ceph-mon[34537]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 792 MiB/s wr, 1.64k op/s 2024-05-11T12:29:15.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:15 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1400532622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:29:15.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:15 smithi050 ceph-mon[34072]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 792 MiB/s wr, 1.64k op/s 2024-05-11T12:29:15.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:15 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1400532622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:29:15.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:15 smithi047 ceph-mon[30375]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 792 MiB/s wr, 1.64k op/s 2024-05-11T12:29:15.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:15 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1400532622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:29:17.256 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:29:17.442 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:17 smithi047 ceph-mon[30375]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 1.63k op/s 2024-05-11T12:29:17.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:17 smithi050 ceph-mon[34072]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 1.63k op/s 2024-05-11T12:29:17.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:17 smithi064 ceph-mon[34537]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 1.63k op/s 2024-05-11T12:29:18.411 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:29:19.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:19 smithi047 ceph-mon[30375]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.65k op/s 2024-05-11T12:29:19.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:19 smithi050 ceph-mon[34072]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.65k op/s 2024-05-11T12:29:19.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:19 smithi064 ceph-mon[34537]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 906 MiB/s wr, 1.65k op/s 2024-05-11T12:29:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:21 smithi050 ceph-mon[34072]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 MiB/s wr, 1.45k op/s 2024-05-11T12:29:21.779 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:21 smithi047 ceph-mon[30375]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 MiB/s wr, 1.45k op/s 2024-05-11T12:29:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:21 smithi064 ceph-mon[34537]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 773 MiB/s wr, 1.45k op/s 2024-05-11T12:29:22.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.814+0000 7f3f4099b700 1 -- 172.21.15.47:0/1292141769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 msgr2=0x7f3f3c11f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:22.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.814+0000 7f3f4099b700 1 --2- 172.21.15.47:0/1292141769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c11f120 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400b0a0 tx=0x7f3f3400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.816+0000 7f3f4099b700 1 -- 172.21.15.47:0/1292141769 shutdown_connections 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.816+0000 7f3f4099b700 1 --2- 172.21.15.47:0/1292141769 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c11f120 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.816+0000 7f3f4099b700 1 --2- 172.21.15.47:0/1292141769 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3c111c40 0x7f3f3c10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.816+0000 7f3f4099b700 1 --2- 172.21.15.47:0/1292141769 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3c1120b0 0x7f3f3c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.816+0000 7f3f4099b700 1 -- 172.21.15.47:0/1292141769 >> 172.21.15.47:0/1292141769 conn(0x7f3f3c06e530 msgr2=0x7f3f3c06e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.818+0000 7f3f4099b700 1 -- 172.21.15.47:0/1292141769 shutdown_connections 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.818+0000 7f3f4099b700 1 -- 172.21.15.47:0/1292141769 wait complete. 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.820+0000 7f3f4099b700 1 Processor -- start 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.820+0000 7f3f4099b700 1 -- start start 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.821+0000 7f3f4099b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3c111c40 0x7f3f3c11b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.821+0000 7f3f4099b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3c1120b0 0x7f3f3c114360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.821+0000 7f3f4099b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c1148a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.821+0000 7f3f4099b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c1225c0 con 0x7f3f3c11cd30 2024-05-11T12:29:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.821+0000 7f3f4099b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c122440 con 0x7f3f3c111c40 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.821+0000 7f3f4099b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c122740 con 0x7f3f3c1120b0 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.822+0000 7f3f3ad9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3c111c40 0x7f3f3c11b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.822+0000 7f3f3a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3c1120b0 0x7f3f3c114360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.823+0000 7f3f3b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c1148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.823+0000 7f3f3b59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c1148a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34192/0 (socket says 172.21.15.47:34192) 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.823+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2524784836 learned_addr learned my addr 172.21.15.47:0/2524784836 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.826+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2524784836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3c1120b0 msgr2=0x7f3f3c114360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.826+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3c1120b0 0x7f3f3c114360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.826+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2524784836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3c111c40 msgr2=0x7f3f3c11b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.826+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3c111c40 0x7f3f3c11b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.826+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2524784836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c114e90 con 0x7f3f3c11cd30 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.836+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c1148a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009fd0 tx=0x7f3f3400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.839+0000 7f3f23fff700 1 -- 172.21.15.47:0/2524784836 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34021020 con 0x7f3f3c11cd30 2024-05-11T12:29:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.839+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c115120 con 0x7f3f3c11cd30 2024-05-11T12:29:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.839+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c119030 con 0x7f3f3c11cd30 2024-05-11T12:29:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.839+0000 7f3f23fff700 1 -- 172.21.15.47:0/2524784836 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f34007b80 con 0x7f3f3c11cd30 2024-05-11T12:29:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.839+0000 7f3f23fff700 1 -- 172.21.15.47:0/2524784836 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340052a0 con 0x7f3f3c11cd30 2024-05-11T12:29:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.851+0000 7f3f23fff700 1 -- 172.21.15.47:0/2524784836 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3f34024020 con 0x7f3f3c11cd30 2024-05-11T12:29:22.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.857+0000 7f3f23fff700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3f240b2a90 0x7f3f240b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.857+0000 7f3f23fff700 1 -- 172.21.15.47:0/2524784836 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f3f3400f070 con 0x7f3f3c11cd30 2024-05-11T12:29:22.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.860+0000 7f3f4ba32b80 1 Processor -- start 2024-05-11T12:29:22.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.860+0000 7f3f4ba32b80 1 -- start start 2024-05-11T12:29:22.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.861+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d035000 0x556f1d0353e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.862+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d02f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.862+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d02ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.863+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556f1cf81940 con 0x556f1d02fb30 2024-05-11T12:29:22.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.863+0000 7f3f3a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d02f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.864+0000 7f3f3a59c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d02f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:51092/0 (socket says 172.21.15.47:51092) 2024-05-11T12:29:22.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.864+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3474905266 learned_addr learned my addr 172.21.15.47:0/3474905266 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:22.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.864+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d02ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.864+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3474905266 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556f1cee6da0 con 0x556f1d035000 2024-05-11T12:29:22.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.864+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3474905266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556f1cf7d900 con 0x556f1d035920 2024-05-11T12:29:22.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.866+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3474905266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d035000 msgr2=0x556f1d0353e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:29:22.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.866+0000 7f3f3a59c700 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d035000 0x556f1d0353e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.866+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3474905266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 msgr2=0x556f1d02ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:22.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.867+0000 7f3f3a59c700 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d02ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.867+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3474905266 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f1cece8d0 con 0x556f1d035920 2024-05-11T12:29:22.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.868+0000 7f3f3ad9d700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3f240b2a90 0x7f3f240b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.868+0000 7f3f3a59c700 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d02f510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009a70 tx=0x7f3f30009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64718c9c23c8285e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.870+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d02ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:29:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.870+0000 7f3f217fa700 1 -- 172.21.15.47:0/3474905266 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30017070 con 0x556f1d035920 2024-05-11T12:29:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.870+0000 7f3f217fa700 1 -- 172.21.15.47:0/3474905266 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f3000ed80 con 0x556f1d035920 2024-05-11T12:29:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.870+0000 7f3f217fa700 1 -- 172.21.15.47:0/3474905266 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f300123d0 con 0x556f1d035920 2024-05-11T12:29:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.872+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3474905266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 msgr2=0x556f1d02f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:22.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.872+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d02f510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009a70 tx=0x7f3f30009d80 comp rx=0 tx=0).stop 2024-05-11T12:29:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.872+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3474905266 shutdown_connections 2024-05-11T12:29:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.872+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d02ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.872+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d035000 0x556f1d0353e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.872+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3474905266 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d02f510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.872+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3474905266 >> 172.21.15.47:0/3474905266 conn(0x556f1cf5bb70 msgr2=0x556f1d02a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.876+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3474905266 shutdown_connections 2024-05-11T12:29:22.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.876+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3474905266 wait complete. 2024-05-11T12:29:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.877+0000 7f3f4ba32b80 1 -- start start 2024-05-11T12:29:22.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.877+0000 7f3f4ba32b80 1 Processor -- start 2024-05-11T12:29:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.878+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d03b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.878+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d035000 0x556f1d038a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.878+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d038f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.878+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556f1d03c440 con 0x556f1d035000 2024-05-11T12:29:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.878+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x556f1cf3aed0 con 0x556f1d02fb30 2024-05-11T12:29:22.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.878+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x556f1d03c5c0 con 0x556f1d035920 2024-05-11T12:29:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.878+0000 7f3f4ba32b80 10 client.0 _ll_register_callbacks cb 0x556f1cf19950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:22.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.883+0000 7f3f3a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d035000 0x556f1d038a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.883+0000 7f3f3a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d035000 0x556f1d038a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34212/0 (socket says 172.21.15.47:34212) 2024-05-11T12:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.883+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3391984166 learned_addr learned my addr 172.21.15.47:0/3391984166 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.883+0000 7f3f3ad9d700 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d03b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.883+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d038f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.885+0000 7f3f3ad9d700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3f240b2a90 0x7f3f240b4f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009ab0 tx=0x7f3f2c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.887+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3391984166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 msgr2=0x556f1d038f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.887+0000 7f3f3a59c700 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d038f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.887+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 msgr2=0x556f1d03b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.887+0000 7f3f3a59c700 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d03b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.887+0000 7f3f3a59c700 1 -- 172.21.15.47:0/3391984166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f1cf38760 con 0x556f1d035000 2024-05-11T12:29:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.890+0000 7f3f3a59c700 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d035000 0x556f1d038a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3f30013d30 tx=0x7f3f30006e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22a0c90e4994d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:22.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.890+0000 7f3f1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.898+0000 7f3f1eff5700 1 -- 172.21.15.47:0/3391984166 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30017040 con 0x556f1d035000 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.898+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f3000ed80 con 0x556f1d035000 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.898+0000 7f3f1eff5700 1 -- 172.21.15.47:0/3391984166 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f3f3001c450 con 0x556f1d035000 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.899+0000 7f3f1eff5700 1 -- 172.21.15.47:0/3391984166 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3f30013880 con 0x556f1d035000 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.899+0000 7f3f4ba32b80 10 client.15537 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.899+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556f1d046ca0 con 0x556f1d035000 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f1eff5700 1 -- 172.21.15.47:0/3391984166 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f3000ed80 con 0x556f1d035000 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f1eff5700 1 -- 172.21.15.47:0/3391984166 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3f30023020 con 0x556f1d035000 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f4ba32b80 10 client.15537 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:22.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f4ba32b80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f4ba32b80 20 client.15537 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d049440 0x556f1d04b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f4ba32b80 4 client.15537 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:29:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.900+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f1cf8f320 con 0x556f1d049440 2024-05-11T12:29:22.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.901+0000 7f3f3ad9d700 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d049440 0x556f1d04b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.911+0000 7f3f1eff5700 10 client.15537 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:22.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.911+0000 7f3f3ad9d700 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d049440 0x556f1d04b880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dfaf38c352bbcee server_cookie=73515b47773bb7b3 in_seq=0 out_seq=0 2024-05-11T12:29:22.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.916+0000 7f3f1eff5700 1 -- 172.21.15.47:0/3391984166 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f1cf8f320 con 0x556f1d049440 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.916+0000 7f3f1eff5700 10 client.15537 handle_command_reply: tid=0 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.916+0000 7f3f4ba32b80 1 client.15537 shutdown 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d035000 msgr2=0x556f1d038a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d035000 0x556f1d038a20 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3f30013d30 tx=0x7f3f30006e90 comp rx=0 tx=0).stop 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 shutdown_connections 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d049440 0x556f1d04b880 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d038f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d03b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/3391984166 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d035000 0x556f1d038a20 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3f30013d30 tx=0x7f3f30006e90 comp rx=0 tx=0).stop 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 >> 172.21.15.47:0/3391984166 conn(0x556f1cf5bb70 msgr2=0x556f1d029660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:22.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.917+0000 7f3f1eff5700 0 client.15537 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:22.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.920+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 shutdown_connections 2024-05-11T12:29:22.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.920+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/3391984166 wait complete. 2024-05-11T12:29:22.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.923+0000 7f3f4ba32b80 20 client.15537 trim_cache size 0 max 16384 2024-05-11T12:29:22.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.973+0000 7f3f4ba32b80 1 Processor -- start 2024-05-11T12:29:22.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.973+0000 7f3f4ba32b80 1 -- start start 2024-05-11T12:29:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.978+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1cf5bb70 0x556f1d02f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.978+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d038f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.979+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d045560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.979+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800b800 con 0x556f1cf5bb70 2024-05-11T12:29:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.979+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800e080 con 0x556f1d02fb30 2024-05-11T12:29:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.979+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800be70 con 0x556f1d035920 2024-05-11T12:29:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.980+0000 7f3f3ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1cf5bb70 0x556f1d02f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.980+0000 7f3f3ad9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1cf5bb70 0x556f1d02f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34222/0 (socket says 172.21.15.47:34222) 2024-05-11T12:29:22.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.980+0000 7f3f3ad9d700 1 -- 172.21.15.47:0/2784859454 learned_addr learned my addr 172.21.15.47:0/2784859454 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.982+0000 7f3f3a59c700 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d038f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.982+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d045560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.986+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2784859454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 msgr2=0x556f1d038f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.986+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d038f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.986+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2784859454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1cf5bb70 msgr2=0x556f1d02f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.986+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1cf5bb70 0x556f1d02f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.986+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2784859454 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f1d029de0 con 0x556f1d035920 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.987+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d045560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f340e1cb0 tx=0x7f3f34009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=872de49e8f438785 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.987+0000 7f3f20ff9700 1 -- 172.21.15.47:0/2784859454 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340e3070 con 0x556f1d035920 2024-05-11T12:29:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.987+0000 7f3f20ff9700 1 -- 172.21.15.47:0/2784859454 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f340ab370 con 0x556f1d035920 2024-05-11T12:29:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.989+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2784859454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 msgr2=0x556f1d045560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.989+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d045560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f340e1cb0 tx=0x7f3f34009040 comp rx=0 tx=0).stop 2024-05-11T12:29:22.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.989+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2784859454 shutdown_connections 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.989+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d035920 0x556f1d045560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.989+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1cf5bb70 0x556f1d02f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.989+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2784859454 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1d02fb30 0x556f1d038f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.989+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2784859454 >> 172.21.15.47:0/2784859454 conn(0x556f1d030270 msgr2=0x556f1d035dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.990+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2784859454 shutdown_connections 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.990+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2784859454 wait complete. 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.990+0000 7f3f4ba32b80 1 -- start start 2024-05-11T12:29:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.991+0000 7f3f4ba32b80 1 Processor -- start 2024-05-11T12:29:22.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.991+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1cf5bb70 0x556f1d0517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.991+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d0418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.992+0000 7f3f4ba32b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d041ec0 0x556f1d050180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:22.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.992+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800c540 con 0x556f1d02fb30 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.992+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800d9b0 con 0x556f1cf5bb70 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.992+0000 7f3f4ba32b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2800cc10 con 0x556f1d041ec0 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.992+0000 7f3f4ba32b80 10 client.0 _ll_register_callbacks cb 0x556f1cf19950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.994+0000 7f3f3b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d041ec0 0x556f1d050180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.994+0000 7f3f3b59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d041ec0 0x556f1d050180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:51122/0 (socket says 172.21.15.47:51122) 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.994+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2771199316 learned_addr learned my addr 172.21.15.47:0/2771199316 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.996+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2771199316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1cf5bb70 msgr2=0x556f1d0517d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:29:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.996+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1cf5bb70 0x556f1d0517d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.996+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2771199316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 msgr2=0x556f1d0418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.996+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d0418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.996+0000 7f3f3b59e700 1 -- 172.21.15.47:0/2771199316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f1d0507c0 con 0x556f1d041ec0 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.996+0000 7f3f3b59e700 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d041ec0 0x556f1d050180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3f340e2020 tx=0x7f3f34009d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=494142ff92276e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.997+0000 7f3f1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.997+0000 7f3f1eff5700 1 -- 172.21.15.47:0/2771199316 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340e3040 con 0x556f1d041ec0 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.997+0000 7f3f1eff5700 1 -- 172.21.15.47:0/2771199316 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f3f340abcd0 con 0x556f1d041ec0 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.998+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f340b2160 con 0x556f1d041ec0 2024-05-11T12:29:23.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.998+0000 7f3f1eff5700 1 -- 172.21.15.47:0/2771199316 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3f340abf60 con 0x556f1d041ec0 2024-05-11T12:29:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.999+0000 7f3f4ba32b80 10 client.25180 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:23.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:22.999+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556f1d0b3710 con 0x556f1d041ec0 2024-05-11T12:29:23.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f1eff5700 1 -- 172.21.15.47:0/2771199316 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f340b4630 con 0x556f1d041ec0 2024-05-11T12:29:23.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f1eff5700 1 -- 172.21.15.47:0/2771199316 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3f34011070 con 0x556f1d041ec0 2024-05-11T12:29:23.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f4ba32b80 10 client.25180 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:23.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f4ba32b80 10 client.25180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:23.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f4ba32b80 20 client.25180 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:23.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d0b3b70 0x556f1d0b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:23.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f4ba32b80 4 client.25180 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:29:23.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.001+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3f340b2160 con 0x556f1d0b3b70 2024-05-11T12:29:23.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.019+0000 7f3f3ad9d700 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d0b3b70 0x556f1d0b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:23.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.031+0000 7f3f3ad9d700 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d0b3b70 0x556f1d0b5f90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cad3b3c2f6321fe8 server_cookie=afa13ab645a91d99 in_seq=0 out_seq=0 2024-05-11T12:29:23.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.031+0000 7f3f1eff5700 10 client.25180 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:23.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.035+0000 7f3f1eff5700 1 -- 172.21.15.47:0/2771199316 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3f340b2160 con 0x556f1d0b3b70 2024-05-11T12:29:23.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.035+0000 7f3f1eff5700 10 client.25180 handle_command_reply: tid=0 2024-05-11T12:29:23.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.035+0000 7f3f4ba32b80 1 client.25180 shutdown 2024-05-11T12:29:23.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.036+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d041ec0 msgr2=0x556f1d050180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:23.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.036+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d041ec0 0x556f1d050180 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3f340e2020 tx=0x7f3f34009d90 comp rx=0 tx=0).stop 2024-05-11T12:29:23.048 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:29:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.042+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 shutdown_connections 2024-05-11T12:29:23.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.042+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556f1d0b3b70 0x556f1d0b5f90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.043+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556f1d041ec0 0x556f1d050180 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.043+0000 7f3f1eff5700 0 client.25180 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.043+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x556f1cf5bb70 0x556f1d0517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.043+0000 7f3f4ba32b80 1 --2- 172.21.15.47:0/2771199316 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556f1d02fb30 0x556f1d0418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.043+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 >> 172.21.15.47:0/2771199316 conn(0x556f1d030270 msgr2=0x556f1d028dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.043+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 shutdown_connections 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.043+0000 7f3f4ba32b80 1 -- 172.21.15.47:0/2771199316 wait complete. 2024-05-11T12:29:23.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.045+0000 7f3f4ba32b80 20 client.25180 trim_cache size 0 max 16384 2024-05-11T12:29:23.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.047+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3f240b2a90 msgr2=0x7f3f240b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:23.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.047+0000 7f3f4099b700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3f240b2a90 0x7f3f240b4f50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009ab0 tx=0x7f3f2c009190 comp rx=0 tx=0).stop 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.047+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 msgr2=0x7f3f3c1148a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.047+0000 7f3f4099b700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c1148a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009fd0 tx=0x7f3f3400ce90 comp rx=0 tx=0).stop 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.048+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 shutdown_connections 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.048+0000 7f3f4099b700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3f3c11cd30 0x7f3f3c1148a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.048+0000 7f3f4099b700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f3f240b2a90 0x7f3f240b4f50 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c009ab0 tx=0x7f3f2c009190 comp rx=0 tx=0).stop 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.048+0000 7f3f4099b700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3f3c111c40 0x7f3f3c11b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.048+0000 7f3f4099b700 1 --2- 172.21.15.47:0/2524784836 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f3f3c1120b0 0x7f3f3c114360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.048+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 >> 172.21.15.47:0/2524784836 conn(0x7f3f3c06e530 msgr2=0x7f3f3c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.051+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 shutdown_connections 2024-05-11T12:29:23.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:23.051+0000 7f3f4099b700 1 -- 172.21.15.47:0/2524784836 wait complete. 2024-05-11T12:29:23.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:23 smithi050 ceph-mon[34072]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 847 MiB/s wr, 1.76k op/s 2024-05-11T12:29:23.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:23 smithi047 ceph-mon[30375]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 847 MiB/s wr, 1.76k op/s 2024-05-11T12:29:23.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:23 smithi064 ceph-mon[34537]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 847 MiB/s wr, 1.76k op/s 2024-05-11T12:29:25.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:25 smithi050 ceph-mon[34072]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 759 MiB/s wr, 1.68k op/s 2024-05-11T12:29:25.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:25 smithi047 ceph-mon[30375]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 759 MiB/s wr, 1.68k op/s 2024-05-11T12:29:25.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:25 smithi064 ceph-mon[34537]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 759 MiB/s wr, 1.68k op/s 2024-05-11T12:29:26.334 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:29:26.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:29:27.447 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:29:27.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:27 smithi050 ceph-mon[34072]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 787 MiB/s wr, 1.78k op/s 2024-05-11T12:29:27.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:27 smithi047 ceph-mon[30375]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 787 MiB/s wr, 1.78k op/s 2024-05-11T12:29:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:27 smithi064 ceph-mon[34537]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 787 MiB/s wr, 1.78k op/s 2024-05-11T12:29:29.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:29 smithi047 ceph-mon[30375]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 876 MiB/s wr, 1.75k op/s 2024-05-11T12:29:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:29 smithi064 ceph-mon[34537]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 876 MiB/s wr, 1.75k op/s 2024-05-11T12:29:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:29 smithi050 ceph-mon[34072]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 876 MiB/s wr, 1.75k op/s 2024-05-11T12:29:31.749 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:31 smithi047 ceph-mon[30375]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 751 MiB/s wr, 1.53k op/s 2024-05-11T12:29:31.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:31 smithi064 ceph-mon[34537]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 751 MiB/s wr, 1.53k op/s 2024-05-11T12:29:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:31 smithi050 ceph-mon[34072]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 751 MiB/s wr, 1.53k op/s 2024-05-11T12:29:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.276+0000 7f183f86d700 1 -- 172.21.15.47:0/649308125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f183810aa90 msgr2=0x7f183810af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.276+0000 7f183f86d700 1 --2- 172.21.15.47:0/649308125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f183810aa90 0x7f183810af10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f18280098f0 tx=0x7f1828009c00 comp rx=0 tx=0).stop 2024-05-11T12:29:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.278+0000 7f183f86d700 1 -- 172.21.15.47:0/649308125 shutdown_connections 2024-05-11T12:29:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.278+0000 7f183f86d700 1 --2- 172.21.15.47:0/649308125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f183811cd30 0x7f183811f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.278+0000 7f183f86d700 1 --2- 172.21.15.47:0/649308125 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1838111d30 0x7f183810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.278+0000 7f183f86d700 1 --2- 172.21.15.47:0/649308125 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f183810aa90 0x7f183810af10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.278+0000 7f183f86d700 1 -- 172.21.15.47:0/649308125 >> 172.21.15.47:0/649308125 conn(0x7f183806e6c0 msgr2=0x7f183806ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:32.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.283+0000 7f183f86d700 1 -- 172.21.15.47:0/649308125 shutdown_connections 2024-05-11T12:29:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.283+0000 7f183f86d700 1 -- 172.21.15.47:0/649308125 wait complete. 2024-05-11T12:29:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.284+0000 7f183f86d700 1 Processor -- start 2024-05-11T12:29:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.285+0000 7f183f86d700 1 -- start start 2024-05-11T12:29:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.286+0000 7f183f86d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f183810aa90 0x7f18381ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.286+0000 7f183f86d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1838111d30 0x7f18381aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.286+0000 7f183f86d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f183811cd30 0x7f18381a89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.286+0000 7f183f86d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f18381226b0 con 0x7f183810aa90 2024-05-11T12:29:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.286+0000 7f183f86d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1838122530 con 0x7f183811cd30 2024-05-11T12:29:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.286+0000 7f183f86d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1838122830 con 0x7f1838111d30 2024-05-11T12:29:32.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.287+0000 7f183de0a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f183811cd30 0x7f18381a89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.287+0000 7f183de0a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f183811cd30 0x7f18381a89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:39936/0 (socket says 172.21.15.47:39936) 2024-05-11T12:29:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.287+0000 7f183de0a700 1 -- 172.21.15.47:0/3278743955 learned_addr learned my addr 172.21.15.47:0/3278743955 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.287+0000 7f183de0a700 1 -- 172.21.15.47:0/3278743955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1838111d30 msgr2=0x7f18381aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.287+0000 7f183de0a700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1838111d30 0x7f18381aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.287+0000 7f183de0a700 1 -- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f183810aa90 msgr2=0x7f18381ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.287+0000 7f183de0a700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f183810aa90 0x7f18381ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.288+0000 7f183de0a700 1 -- 172.21.15.47:0/3278743955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18381a9130 con 0x7f183811cd30 2024-05-11T12:29:32.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.292+0000 7f183de0a700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f183811cd30 0x7f18381a89d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f183000b9e0 tx=0x7f183000bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.293+0000 7f182e7fc700 1 -- 172.21.15.47:0/3278743955 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830019070 con 0x7f183811cd30 2024-05-11T12:29:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.293+0000 7f182e7fc700 1 -- 172.21.15.47:0/3278743955 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1830014440 con 0x7f183811cd30 2024-05-11T12:29:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.293+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18381a9420 con 0x7f183811cd30 2024-05-11T12:29:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.293+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18381b5690 con 0x7f183811cd30 2024-05-11T12:29:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.293+0000 7f182e7fc700 1 -- 172.21.15.47:0/3278743955 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183000ce60 con 0x7f183811cd30 2024-05-11T12:29:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.302+0000 7f184a103b80 1 Processor -- start 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.311+0000 7f184a103b80 1 -- start start 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.311+0000 7f184a103b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.311+0000 7f184a103b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.311+0000 7f184a103b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.311+0000 7f184a103b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563f9e71c9c0 con 0x563f9e7cee00 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.311+0000 7f184a103b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f9e681e20 con 0x563f9e7cf720 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.311+0000 7f184a103b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f9e718980 con 0x563f9e7cabb0 2024-05-11T12:29:32.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.312+0000 7f182e7fc700 1 -- 172.21.15.47:0/3278743955 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f183001c070 con 0x7f183811cd30 2024-05-11T12:29:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.312+0000 7f183ce08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.312+0000 7f183ce08700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:59772/0 (socket says 172.21.15.47:59772) 2024-05-11T12:29:32.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.313+0000 7f183ce08700 1 -- 172.21.15.47:0/1894371703 learned_addr learned my addr 172.21.15.47:0/1894371703 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.313+0000 7f183d609700 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.314+0000 7f183ce08700 1 -- 172.21.15.47:0/1894371703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 msgr2=0x563f9e7ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.314+0000 7f183ce08700 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.314+0000 7f183ce08700 1 -- 172.21.15.47:0/1894371703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 msgr2=0x563f9e7cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.314+0000 7f183ce08700 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.314+0000 7f183ce08700 1 -- 172.21.15.47:0/1894371703 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f9e669950 con 0x563f9e7cabb0 2024-05-11T12:29:32.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.315+0000 7f183ce08700 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7cb010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1828009fd0 tx=0x7f1828009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40f92d804a7c053 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.315+0000 7f182e7fc700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f18240b2a90 0x7f18240b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.315+0000 7f18237fe700 1 -- 172.21.15.47:0/1894371703 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828026070 con 0x563f9e7cabb0 2024-05-11T12:29:32.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.316+0000 7f18237fe700 1 -- 172.21.15.47:0/1894371703 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f182800f910 con 0x563f9e7cabb0 2024-05-11T12:29:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.316+0000 7f182e7fc700 1 -- 172.21.15.47:0/3278743955 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f18300e1600 con 0x7f183811cd30 2024-05-11T12:29:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.317+0000 7f184a103b80 1 -- 172.21.15.47:0/1894371703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 msgr2=0x563f9e7cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.317+0000 7f184a103b80 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7cb010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1828009fd0 tx=0x7f1828009590 comp rx=0 tx=0).stop 2024-05-11T12:29:32.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.320+0000 7f183d609700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f18240b2a90 0x7f18240b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.322+0000 7f183d609700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f18240b2a90 0x7f18240b4f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f18340095a0 tx=0x7f1834009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.322+0000 7f184a103b80 1 -- 172.21.15.47:0/1894371703 shutdown_connections 2024-05-11T12:29:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.322+0000 7f184a103b80 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7cb010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.322+0000 7f184a103b80 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.322+0000 7f184a103b80 1 --2- 172.21.15.47:0/1894371703 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.322+0000 7f184a103b80 1 -- 172.21.15.47:0/1894371703 >> 172.21.15.47:0/1894371703 conn(0x563f9e6f6bf0 msgr2=0x563f9e7c5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.326+0000 7f184a103b80 1 -- 172.21.15.47:0/1894371703 shutdown_connections 2024-05-11T12:29:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.326+0000 7f184a103b80 1 -- 172.21.15.47:0/1894371703 wait complete. 2024-05-11T12:29:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.327+0000 7f184a103b80 1 -- start start 2024-05-11T12:29:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.327+0000 7f184a103b80 1 Processor -- start 2024-05-11T12:29:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.329+0000 7f184a103b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cabb0 0x563f9e7d64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.329+0000 7f184a103b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.329+0000 7f184a103b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cf720 0x563f9e7d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.337+0000 7f183d609700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cabb0 0x563f9e7d64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.330+0000 7f183ce08700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.335+0000 7f183de0a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cf720 0x563f9e7d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.337+0000 7f183de0a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cf720 0x563f9e7d3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:59786/0 (socket says 172.21.15.47:59786) 2024-05-11T12:29:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.337+0000 7f183de0a700 1 -- 172.21.15.47:0/1006819495 learned_addr learned my addr 172.21.15.47:0/1006819495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:32.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.337+0000 7f183ce08700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7d3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50160/0 (socket says 172.21.15.47:50160) 2024-05-11T12:29:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.338+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x563f9e7e82f0 con 0x563f9e7cee00 2024-05-11T12:29:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.338+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x563f9e6d5f50 con 0x563f9e7cabb0 2024-05-11T12:29:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.338+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x563f9e7e8470 con 0x563f9e7cf720 2024-05-11T12:29:32.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.338+0000 7f184a103b80 10 client.0 _ll_register_callbacks cb 0x563f9e6b49d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:32.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.342+0000 7f183ce08700 1 -- 172.21.15.47:0/1006819495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cf720 msgr2=0x563f9e7d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.342+0000 7f183ce08700 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cf720 0x563f9e7d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.342+0000 7f183ce08700 1 -- 172.21.15.47:0/1006819495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cabb0 msgr2=0x563f9e7d64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.342+0000 7f183ce08700 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cabb0 0x563f9e7d64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.342+0000 7f183ce08700 1 -- 172.21.15.47:0/1006819495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f9e6d37e0 con 0x563f9e7cee00 2024-05-11T12:29:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.343+0000 7f183ce08700 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7d3850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f18280098f0 tx=0x7f182800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a61ada37c45927 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.343+0000 7f1820ff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:29:32.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.344+0000 7f1820ff9700 1 -- 172.21.15.47:0/1006819495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828026030 con 0x563f9e7cee00 2024-05-11T12:29:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.344+0000 7f1820ff9700 1 -- 172.21.15.47:0/1006819495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f182801e040 con 0x563f9e7cee00 2024-05-11T12:29:32.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.344+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f182800f910 con 0x563f9e7cee00 2024-05-11T12:29:32.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.345+0000 7f1820ff9700 1 -- 172.21.15.47:0/1006819495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f182800f670 con 0x563f9e7cee00 2024-05-11T12:29:32.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.348+0000 7f184a103b80 10 client.15543 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:32.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.348+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563f9e7f2b30 con 0x563f9e7cee00 2024-05-11T12:29:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.352+0000 7f1820ff9700 1 -- 172.21.15.47:0/1006819495 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1828028050 con 0x563f9e7cee00 2024-05-11T12:29:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.352+0000 7f1820ff9700 1 -- 172.21.15.47:0/1006819495 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1828021b10 con 0x563f9e7cee00 2024-05-11T12:29:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.353+0000 7f184a103b80 10 client.15543 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.353+0000 7f184a103b80 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.354+0000 7f184a103b80 20 client.15543 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.354+0000 7f184a103b80 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e7f5000 0x563f9e7f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.354+0000 7f184a103b80 4 client.15543 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:29:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.354+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f182800f910 con 0x563f9e7f5000 2024-05-11T12:29:32.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.355+0000 7f183d609700 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e7f5000 0x563f9e7f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.359+0000 7f183d609700 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e7f5000 0x563f9e7f73f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1254934693d07db server_cookie=eb84af60fa1d18c2 in_seq=0 out_seq=0 2024-05-11T12:29:32.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.359+0000 7f1820ff9700 10 client.15543 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:32.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.364+0000 7f1820ff9700 1 -- 172.21.15.47:0/1006819495 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f182800f910 con 0x563f9e7f5000 2024-05-11T12:29:32.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.364+0000 7f1820ff9700 10 client.15543 handle_command_reply: tid=0 2024-05-11T12:29:32.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.364+0000 7f184a103b80 1 client.15543 shutdown 2024-05-11T12:29:32.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.364+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 msgr2=0x563f9e7d3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.364+0000 7f184a103b80 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7d3850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f18280098f0 tx=0x7f182800b560 comp rx=0 tx=0).stop 2024-05-11T12:29:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.365+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 shutdown_connections 2024-05-11T12:29:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.365+0000 7f184a103b80 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e7f5000 0x563f9e7f73f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.365+0000 7f184a103b80 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cf720 0x563f9e7d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.365+0000 7f184a103b80 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cabb0 0x563f9e7d64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.365+0000 7f184a103b80 1 --2- 172.21.15.47:0/1006819495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7d3850 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.365+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 >> 172.21.15.47:0/1006819495 conn(0x563f9e6f6bf0 msgr2=0x563f9e7bf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:32.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.365+0000 7f1820ff9700 0 client.15543 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:32.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.366+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 shutdown_connections 2024-05-11T12:29:32.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.366+0000 7f184a103b80 1 -- 172.21.15.47:0/1006819495 wait complete. 2024-05-11T12:29:32.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.367+0000 7f184a103b80 20 client.15543 trim_cache size 0 max 16384 2024-05-11T12:29:32.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.418+0000 7f184a103b80 1 Processor -- start 2024-05-11T12:29:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.419+0000 7f184a103b80 1 -- start start 2024-05-11T12:29:32.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.419+0000 7f184a103b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7d3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.419+0000 7f184a103b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.419+0000 7f184a103b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.419+0000 7f184a103b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f181c00b830 con 0x563f9e7cee00 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.419+0000 7f184a103b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f181c00e0b0 con 0x563f9e7cf720 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.419+0000 7f184a103b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f181c00bea0 con 0x563f9e7cabb0 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.420+0000 7f183ce08700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.420+0000 7f183ce08700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7bf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50168/0 (socket says 172.21.15.47:50168) 2024-05-11T12:29:32.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.420+0000 7f183ce08700 1 -- 172.21.15.47:0/2959412238 learned_addr learned my addr 172.21.15.47:0/2959412238 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:32.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.420+0000 7f183de0a700 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.423+0000 7f183d609700 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7d3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.425+0000 7f183ce08700 1 -- 172.21.15.47:0/2959412238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 msgr2=0x563f9e7d3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.425+0000 7f183ce08700 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7d3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.425+0000 7f183ce08700 1 -- 172.21.15.47:0/2959412238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 msgr2=0x563f9e7f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.425+0000 7f183ce08700 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.425+0000 7f183ce08700 1 -- 172.21.15.47:0/2959412238 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f9e7bff00 con 0x563f9e7cee00 2024-05-11T12:29:32.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.436+0000 7f183ce08700 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7bf9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1828005040 tx=0x7f1828009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736b793fdc1e8b23 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.445+0000 7f1822ffd700 1 -- 172.21.15.47:0/2959412238 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828026030 con 0x563f9e7cee00 2024-05-11T12:29:32.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.445+0000 7f1822ffd700 1 -- 172.21.15.47:0/2959412238 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1828017db0 con 0x563f9e7cee00 2024-05-11T12:29:32.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.445+0000 7f1822ffd700 1 -- 172.21.15.47:0/2959412238 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18280053d0 con 0x563f9e7cee00 2024-05-11T12:29:32.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.450+0000 7f184a103b80 1 -- 172.21.15.47:0/2959412238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 msgr2=0x563f9e7bf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.450+0000 7f184a103b80 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7bf9c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1828005040 tx=0x7f1828009590 comp rx=0 tx=0).stop 2024-05-11T12:29:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.454+0000 7f184a103b80 1 -- 172.21.15.47:0/2959412238 shutdown_connections 2024-05-11T12:29:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.454+0000 7f184a103b80 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cf720 0x563f9e7f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.454+0000 7f184a103b80 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7d3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.454+0000 7f184a103b80 1 --2- 172.21.15.47:0/2959412238 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cee00 0x563f9e7bf9c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.454+0000 7f184a103b80 1 -- 172.21.15.47:0/2959412238 >> 172.21.15.47:0/2959412238 conn(0x563f9e6f6bf0 msgr2=0x563f9e7f07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.456+0000 7f184a103b80 1 -- 172.21.15.47:0/2959412238 shutdown_connections 2024-05-11T12:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.456+0000 7f184a103b80 1 -- 172.21.15.47:0/2959412238 wait complete. 2024-05-11T12:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.457+0000 7f184a103b80 1 -- start start 2024-05-11T12:29:32.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.457+0000 7f184a103b80 1 Processor -- start 2024-05-11T12:29:32.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.462+0000 7f184a103b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.462+0000 7f184a103b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cee00 0x563f9e7f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.462+0000 7f184a103b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cf720 0x563f9e85e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.462+0000 7f184a103b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f181c00c570 con 0x563f9e7cf720 2024-05-11T12:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.462+0000 7f184a103b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f181c00d9e0 con 0x563f9e7cee00 2024-05-11T12:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.462+0000 7f184a103b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f181c00cc40 con 0x563f9e7cabb0 2024-05-11T12:29:32.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.462+0000 7f184a103b80 10 client.0 _ll_register_callbacks cb 0x563f9e6b49d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:32.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.465+0000 7f183ce08700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cee00 0x563f9e7f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.466+0000 7f183de0a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cf720 0x563f9e85e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.466+0000 7f183de0a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cf720 0x563f9e85e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50182/0 (socket says 172.21.15.47:50182) 2024-05-11T12:29:32.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.466+0000 7f183de0a700 1 -- 172.21.15.47:0/2342268389 learned_addr learned my addr 172.21.15.47:0/2342268389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:32.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.466+0000 7f183d609700 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.471+0000 7f183ce08700 1 -- 172.21.15.47:0/2342268389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 msgr2=0x563f9e7f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.471+0000 7f183ce08700 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.471+0000 7f183ce08700 1 -- 172.21.15.47:0/2342268389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cf720 msgr2=0x563f9e85e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.471+0000 7f183ce08700 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cf720 0x563f9e85e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.471+0000 7f183ce08700 1 -- 172.21.15.47:0/2342268389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f9e85ea50 con 0x563f9e7cee00 2024-05-11T12:29:32.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.473+0000 7f183ce08700 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cee00 0x563f9e7f6500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x563f9e7cab00 tx=0x7f1828017f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=680e0dc8c51f486f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:32.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.474+0000 7f1820ff9700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.474+0000 7f1820ff9700 1 -- 172.21.15.47:0/2342268389 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828026040 con 0x563f9e7cee00 2024-05-11T12:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.475+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1828017db0 con 0x563f9e7cee00 2024-05-11T12:29:32.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.475+0000 7f1820ff9700 1 -- 172.21.15.47:0/2342268389 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1828003680 con 0x563f9e7cee00 2024-05-11T12:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.476+0000 7f1820ff9700 1 -- 172.21.15.47:0/2342268389 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f182803e050 con 0x563f9e7cee00 2024-05-11T12:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.476+0000 7f184a103b80 10 client.25145 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:32.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.476+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563f9e85fc40 con 0x563f9e7cee00 2024-05-11T12:29:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f1820ff9700 1 -- 172.21.15.47:0/2342268389 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1828017db0 con 0x563f9e7cee00 2024-05-11T12:29:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f1820ff9700 1 -- 172.21.15.47:0/2342268389 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1828021410 con 0x563f9e7cee00 2024-05-11T12:29:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f184a103b80 10 client.25145 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:32.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f184a103b80 10 client.25145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f184a103b80 20 client.25145 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f184a103b80 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e860180 0x563f9e8625a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f184a103b80 4 client.25145 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:29:32.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.480+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563f9e72a3a0 con 0x563f9e860180 2024-05-11T12:29:32.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.488+0000 7f183d609700 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e860180 0x563f9e8625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:32.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.505+0000 7f183d609700 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e860180 0x563f9e8625a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cf869fdfe1d6cf server_cookie=cd8c852652c493f6 in_seq=0 out_seq=0 2024-05-11T12:29:32.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.505+0000 7f1820ff9700 10 client.25145 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:32.702 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "4632259c-0563-4d49-aee2-012fe65553ba", 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.699+0000 7f1820ff9700 1 -- 172.21.15.47:0/2342268389 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563f9e72a3a0 con 0x563f9e860180 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.699+0000 7f1820ff9700 10 client.25145 handle_command_reply: tid=0 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.699+0000 7f184a103b80 1 client.25145 shutdown 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cee00 msgr2=0x563f9e7f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cee00 0x563f9e7f6500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x563f9e7cab00 tx=0x7f1828017f60 comp rx=0 tx=0).stop 2024-05-11T12:29:32.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 shutdown_connections 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x563f9e860180 0x563f9e8625a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x563f9e7cf720 0x563f9e85e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x563f9e7cabb0 0x563f9e7f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 --2- 172.21.15.47:0/2342268389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x563f9e7cee00 0x563f9e7f6500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 >> 172.21.15.47:0/2342268389 conn(0x563f9e6f6bf0 msgr2=0x563f9e7f1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f1820ff9700 0 client.25145 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 shutdown_connections 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.700+0000 7f184a103b80 1 -- 172.21.15.47:0/2342268389 wait complete. 2024-05-11T12:29:32.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.701+0000 7f184a103b80 20 client.25145 trim_cache size 0 max 16384 2024-05-11T12:29:32.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f18240b2a90 msgr2=0x7f18240b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f18240b2a90 0x7f18240b4f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f18340095a0 tx=0x7f1834009510 comp rx=0 tx=0).stop 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f183811cd30 msgr2=0x7f18381a89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f183811cd30 0x7f18381a89d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f183000b9e0 tx=0x7f183000bda0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 shutdown_connections 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f18240b2a90 0x7f18240b4f50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f183811cd30 0x7f18381a89d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f183810aa90 0x7f18381ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 --2- 172.21.15.47:0/3278743955 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1838111d30 0x7f18381aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 >> 172.21.15.47:0/3278743955 conn(0x7f183806e6c0 msgr2=0x7f183811ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 shutdown_connections 2024-05-11T12:29:32.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:32.713+0000 7f183f86d700 1 -- 172.21.15.47:0/3278743955 wait complete. 2024-05-11T12:29:33.779 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:33 smithi047 ceph-mon[30375]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 1.76k op/s 2024-05-11T12:29:33.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:33 smithi064 ceph-mon[34537]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 1.76k op/s 2024-05-11T12:29:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:33 smithi050 ceph-mon[34072]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 1.76k op/s 2024-05-11T12:29:34.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:34 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:29:34.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:34 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:29:34.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:34 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:29:34.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:34 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:29:34.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:34 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:29:34.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:34 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:29:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:34 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:29:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:34 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:29:34.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:34 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:29:35.361 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:29:35.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:35 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:29:35.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:35 smithi047 ceph-mon[30375]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 780 MiB/s wr, 1.57k op/s 2024-05-11T12:29:35.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:35 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:29:35.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:35 smithi064 ceph-mon[34537]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 780 MiB/s wr, 1.57k op/s 2024-05-11T12:29:35.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:35 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:29:35.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:35 smithi050 ceph-mon[34072]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 780 MiB/s wr, 1.57k op/s 2024-05-11T12:29:36.205 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:29:37.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:37 smithi064 ceph-mon[34537]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 1.67k op/s 2024-05-11T12:29:37.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:37 smithi050 ceph-mon[34072]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 1.67k op/s 2024-05-11T12:29:38.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:37 smithi047 ceph-mon[30375]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 810 MiB/s wr, 1.67k op/s 2024-05-11T12:29:39.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:39 smithi047 ceph-mon[30375]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.79k op/s 2024-05-11T12:29:39.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:39 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:29:39.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:39 smithi064 ceph-mon[34537]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.79k op/s 2024-05-11T12:29:39.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:39 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:29:39.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:39 smithi050 ceph-mon[34072]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.79k op/s 2024-05-11T12:29:39.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:39 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:29:40.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.817+0000 7f4638dcf700 1 -- 172.21.15.47:0/663030652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4634111d30 msgr2=0x7f463410a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:40.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.817+0000 7f4638dcf700 1 --2- 172.21.15.47:0/663030652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4634111d30 0x7f463410a550 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f461c009990 tx=0x7f461c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.818+0000 7f4638dcf700 1 -- 172.21.15.47:0/663030652 shutdown_connections 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.818+0000 7f4638dcf700 1 --2- 172.21.15.47:0/663030652 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f463411cd30 0x7f463411f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.818+0000 7f4638dcf700 1 --2- 172.21.15.47:0/663030652 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4634111d30 0x7f463410a550 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.818+0000 7f4638dcf700 1 --2- 172.21.15.47:0/663030652 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f463410aa90 0x7f463410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.818+0000 7f4638dcf700 1 -- 172.21.15.47:0/663030652 >> 172.21.15.47:0/663030652 conn(0x7f463406e6c0 msgr2=0x7f463406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.819+0000 7f4638dcf700 1 -- 172.21.15.47:0/663030652 shutdown_connections 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.820+0000 7f4638dcf700 1 -- 172.21.15.47:0/663030652 wait complete. 2024-05-11T12:29:40.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 Processor -- start 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 -- start start 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f463410aa90 0x7f46341161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4634111d30 0x7f4634116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f463411cd30 0x7f463411a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f46341129e0 con 0x7f463410aa90 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4634112860 con 0x7f463411cd30 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.822+0000 7f4638dcf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4634112b60 con 0x7f4634111d30 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.823+0000 7f4629d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4634111d30 0x7f4634116700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.823+0000 7f463259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f463410aa90 0x7f46341161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.823+0000 7f463259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f463410aa90 0x7f46341161c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60686/0 (socket says 172.21.15.47:60686) 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.823+0000 7f463259c700 1 -- 172.21.15.47:0/4119802024 learned_addr learned my addr 172.21.15.47:0/4119802024 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.824+0000 7f463259c700 1 -- 172.21.15.47:0/4119802024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4634111d30 msgr2=0x7f4634116700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:40.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.824+0000 7f463259c700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4634111d30 0x7f4634116700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.824+0000 7f463259c700 1 -- 172.21.15.47:0/4119802024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f463411cd30 msgr2=0x7f463411a9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.824+0000 7f463259c700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f463411cd30 0x7f463411a9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.824+0000 7f463259c700 1 -- 172.21.15.47:0/4119802024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463411b0f0 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.824+0000 7f463259c700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f463410aa90 0x7f46341161c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c00ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.825+0000 7f462bfff700 1 -- 172.21.15.47:0/4119802024 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c026070 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.825+0000 7f462bfff700 1 -- 172.21.15.47:0/4119802024 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c003680 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.825+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463411b380 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.825+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46341bdec0 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.829+0000 7f462bfff700 1 -- 172.21.15.47:0/4119802024 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c021670 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.833+0000 7f462bfff700 1 -- 172.21.15.47:0/4119802024 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f461c029070 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.844+0000 7f462bfff700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f46200b2a00 0x7f46200b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.845+0000 7f462bfff700 1 -- 172.21.15.47:0/4119802024 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f461c00fa30 con 0x7f463410aa90 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.846+0000 7f4643665b80 1 Processor -- start 2024-05-11T12:29:40.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.846+0000 7f4629d9b700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f46200b2a00 0x7f46200b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.848+0000 7f4629d9b700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f46200b2a00 0x7f46200b4ec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4634117710 tx=0x7f462400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.849+0000 7f4643665b80 1 -- start start 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.850+0000 7f4643665b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.850+0000 7f4643665b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a1480a0 0x55708a148520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.850+0000 7f4643665b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a154890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.850+0000 7f4643665b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55708a09a860 con 0x55708a148a60 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.851+0000 7f4643665b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55708a0397c0 con 0x55708a1480a0 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.851+0000 7f4643665b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55708a096820 con 0x55708a14dad0 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.851+0000 7f4632d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.855+0000 7f463259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.855+0000 7f4629d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a1480a0 0x55708a148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.856+0000 7f463259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a14deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49080/0 (socket says 172.21.15.47:49080) 2024-05-11T12:29:40.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.856+0000 7f463259c700 1 -- 172.21.15.47:0/2046344878 learned_addr learned my addr 172.21.15.47:0/2046344878 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.861+0000 7f463259c700 1 -- 172.21.15.47:0/2046344878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a1480a0 msgr2=0x55708a148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.861+0000 7f463259c700 1 --2- 172.21.15.47:0/2046344878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a1480a0 0x55708a148520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.861+0000 7f463259c700 1 -- 172.21.15.47:0/2046344878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 msgr2=0x55708a154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.861+0000 7f463259c700 1 --2- 172.21.15.47:0/2046344878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.861+0000 7f463259c700 1 -- 172.21.15.47:0/2046344878 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557089fe78d0 con 0x55708a14dad0 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.866+0000 7f463259c700 1 --2- 172.21.15.47:0/2046344878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a14deb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f461c0a8330 tx=0x7f461c0aaae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f8f18c9394751e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.870+0000 7f4628d99700 1 -- 172.21.15.47:0/2046344878 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0aabd0 con 0x55708a14dad0 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.870+0000 7f4628d99700 1 -- 172.21.15.47:0/2046344878 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c0ab7c0 con 0x55708a14dad0 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.870+0000 7f4628d99700 1 -- 172.21.15.47:0/2046344878 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0b3a00 con 0x55708a14dad0 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.871+0000 7f4643665b80 1 -- 172.21.15.47:0/2046344878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 msgr2=0x55708a14deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.871+0000 7f4643665b80 1 --2- 172.21.15.47:0/2046344878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a14deb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f461c0a8330 tx=0x7f461c0aaae0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.877+0000 7f4643665b80 1 -- 172.21.15.47:0/2046344878 shutdown_connections 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.877+0000 7f4643665b80 1 --2- 172.21.15.47:0/2046344878 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.877+0000 7f4643665b80 1 --2- 172.21.15.47:0/2046344878 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a14deb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.877+0000 7f4643665b80 1 --2- 172.21.15.47:0/2046344878 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a1480a0 0x55708a148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.877+0000 7f4643665b80 1 -- 172.21.15.47:0/2046344878 >> 172.21.15.47:0/2046344878 conn(0x55708a074e60 msgr2=0x55708a142990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.881+0000 7f4643665b80 1 -- 172.21.15.47:0/2046344878 shutdown_connections 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.883+0000 7f4643665b80 1 -- 172.21.15.47:0/2046344878 wait complete. 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.884+0000 7f4643665b80 1 -- start start 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.884+0000 7f4643665b80 1 Processor -- start 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.891+0000 7f4643665b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a1544e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.891+0000 7f4643665b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a151880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.891+0000 7f4643665b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a14dad0 0x55708a151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.891+0000 7f4643665b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55708a14e5e0 con 0x55708a148a60 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.891+0000 7f4643665b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55708a000c30 con 0x55708a14dad0 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.891+0000 7f4643665b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55708a154dd0 con 0x55708a1480a0 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.892+0000 7f4643665b80 10 client.0 _ll_register_callbacks cb 0x55708a032920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:40.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.892+0000 7f463259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a1544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.892+0000 7f463259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a1544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49094/0 (socket says 172.21.15.47:49094) 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.892+0000 7f463259c700 1 -- 172.21.15.47:0/3332223365 learned_addr learned my addr 172.21.15.47:0/3332223365 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.893+0000 7f4629d9b700 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.893+0000 7f463259c700 1 -- 172.21.15.47:0/3332223365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a14dad0 msgr2=0x55708a151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.893+0000 7f463259c700 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a14dad0 0x55708a151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.893+0000 7f463259c700 1 -- 172.21.15.47:0/3332223365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 msgr2=0x55708a151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.893+0000 7f463259c700 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.893+0000 7f463259c700 1 -- 172.21.15.47:0/3332223365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55708a036690 con 0x55708a1480a0 2024-05-11T12:29:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.900+0000 7f463259c700 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a1544e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f461c00b5c0 tx=0x7f461c0aad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7833c824be648d3d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.900+0000 7f4619ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:29:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.902+0000 7f4619ffb700 1 -- 172.21.15.47:0/3332223365 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0c9020 con 0x55708a1480a0 2024-05-11T12:29:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.902+0000 7f4619ffb700 1 -- 172.21.15.47:0/3332223365 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f461c0b3a00 con 0x55708a1480a0 2024-05-11T12:29:40.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.902+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c0ab7c0 con 0x55708a1480a0 2024-05-11T12:29:40.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.906+0000 7f4619ffb700 1 -- 172.21.15.47:0/3332223365 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f461c0b3c00 con 0x55708a1480a0 2024-05-11T12:29:40.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.906+0000 7f4643665b80 10 client.25204 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:40.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.906+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55708a15f190 con 0x55708a1480a0 2024-05-11T12:29:40.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.906+0000 7f4619ffb700 1 -- 172.21.15.47:0/3332223365 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c0c01d0 con 0x55708a1480a0 2024-05-11T12:29:40.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.906+0000 7f4619ffb700 1 -- 172.21.15.47:0/3332223365 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f461c00f620 con 0x55708a1480a0 2024-05-11T12:29:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.907+0000 7f4643665b80 10 client.25204 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.907+0000 7f4643665b80 10 client.25204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.907+0000 7f4643665b80 20 client.25204 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.907+0000 7f4643665b80 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a161660 0x55708a163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.907+0000 7f4643665b80 4 client.25204 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:29:40.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.907+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f461c0ab7c0 con 0x55708a161660 2024-05-11T12:29:40.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.925+0000 7f4629d9b700 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a161660 0x55708a163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.938+0000 7f4619ffb700 10 client.25204 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.938+0000 7f4629d9b700 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a161660 0x55708a163a50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a644ae373517394 server_cookie=cd63b5cae4dcd0d9 in_seq=0 out_seq=0 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4619ffb700 1 -- 172.21.15.47:0/3332223365 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f461c0ab7c0 con 0x55708a161660 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4619ffb700 10 client.25204 handle_command_reply: tid=0 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 client.25204 shutdown 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 msgr2=0x55708a1544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a1544e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f461c00b5c0 tx=0x7f461c0aad00 comp rx=0 tx=0).stop 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 shutdown_connections 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a161660 0x55708a163a50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a14dad0 0x55708a151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a1544e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4619ffb700 0 client.25204 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:40.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 --2- 172.21.15.47:0/3332223365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a148a60 0x55708a151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:40.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.940+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 >> 172.21.15.47:0/3332223365 conn(0x55708a074e60 msgr2=0x55708a13d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:40.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.941+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 shutdown_connections 2024-05-11T12:29:40.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.941+0000 7f4643665b80 1 -- 172.21.15.47:0/3332223365 wait complete. 2024-05-11T12:29:40.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.941+0000 7f4643665b80 20 client.25204 trim_cache size 0 max 16384 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.992+0000 7f4643665b80 1 Processor -- start 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.992+0000 7f4643665b80 1 -- start start 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.993+0000 7f4643665b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a151e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.993+0000 7f4643665b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f4643665b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a14dad0 0x55708a13d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f4643665b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f461400b7a0 con 0x55708a14dad0 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f4643665b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f461400e020 con 0x55708a148a60 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f4643665b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f461400be10 con 0x55708a1480a0 2024-05-11T12:29:40.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f463259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f463259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49098/0 (socket says 172.21.15.47:49098) 2024-05-11T12:29:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f463259c700 1 -- 172.21.15.47:0/53653309 learned_addr learned my addr 172.21.15.47:0/53653309 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:40.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.994+0000 7f4629d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:40.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.997+0000 7f4632d9d700 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a14dad0 0x55708a13d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.998+0000 7f4629d9b700 1 -- 172.21.15.47:0/53653309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 msgr2=0x55708a151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.998+0000 7f4629d9b700 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.998+0000 7f4629d9b700 1 -- 172.21.15.47:0/53653309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a14dad0 msgr2=0x55708a13d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:41.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.998+0000 7f4629d9b700 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a14dad0 0x55708a13d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.998+0000 7f4629d9b700 1 -- 172.21.15.47:0/53653309 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55708a13f590 con 0x55708a148a60 2024-05-11T12:29:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.998+0000 7f4629d9b700 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15dc10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55708a1642c0 tx=0x7f4624016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=754db9de2e965663 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:40.999+0000 7f461bfff700 1 -- 172.21.15.47:0/53653309 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624016070 con 0x55708a148a60 2024-05-11T12:29:41.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.000+0000 7f461bfff700 1 -- 172.21.15.47:0/53653309 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4624010810 con 0x55708a148a60 2024-05-11T12:29:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.001+0000 7f4643665b80 1 -- 172.21.15.47:0/53653309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 msgr2=0x55708a15dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:41.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.001+0000 7f4643665b80 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15dc10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55708a1642c0 tx=0x7f4624016000 comp rx=0 tx=0).stop 2024-05-11T12:29:41.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.003+0000 7f4643665b80 1 -- 172.21.15.47:0/53653309 shutdown_connections 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.003+0000 7f4643665b80 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a14dad0 0x55708a13d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.003+0000 7f4643665b80 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a1480a0 0x55708a151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.003+0000 7f4643665b80 1 --2- 172.21.15.47:0/53653309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15dc10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.003+0000 7f4643665b80 1 -- 172.21.15.47:0/53653309 >> 172.21.15.47:0/53653309 conn(0x55708a074e60 msgr2=0x55708a13cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.004+0000 7f4643665b80 1 -- 172.21.15.47:0/53653309 shutdown_connections 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.004+0000 7f4643665b80 1 -- 172.21.15.47:0/53653309 wait complete. 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.004+0000 7f4643665b80 1 -- start start 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 1 Processor -- start 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a1480a0 0x55708a15a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a156f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f461400c4e0 con 0x55708a1480a0 2024-05-11T12:29:41.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f461400d950 con 0x55708a148a60 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f461400cbb0 con 0x55708a14dad0 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.005+0000 7f4643665b80 10 client.0 _ll_register_callbacks cb 0x55708a032920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.006+0000 7f463259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a1480a0 0x55708a15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.006+0000 7f463259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a1480a0 0x55708a15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60718/0 (socket says 172.21.15.47:60718) 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.006+0000 7f463259c700 1 -- 172.21.15.47:0/548252945 learned_addr learned my addr 172.21.15.47:0/548252945 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.007+0000 7f4632d9d700 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.007+0000 7f4629d9b700 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.007+0000 7f4629d9b700 1 -- 172.21.15.47:0/548252945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 msgr2=0x55708a156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.007+0000 7f4629d9b700 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.007+0000 7f4629d9b700 1 -- 172.21.15.47:0/548252945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a1480a0 msgr2=0x55708a15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:41.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.007+0000 7f4629d9b700 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a1480a0 0x55708a15a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.007+0000 7f4629d9b700 1 -- 172.21.15.47:0/548252945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55708a157730 con 0x55708a148a60 2024-05-11T12:29:41.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.008+0000 7f4629d9b700 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15a680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55708a148000 tx=0x7f4624027ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e7557f33a3feb server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.008+0000 7f4619ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:29:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.009+0000 7f4619ffb700 1 -- 172.21.15.47:0/548252945 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4624010dd0 con 0x55708a148a60 2024-05-11T12:29:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.010+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4624010810 con 0x55708a148a60 2024-05-11T12:29:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.010+0000 7f4619ffb700 1 -- 172.21.15.47:0/548252945 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f462402f410 con 0x55708a148a60 2024-05-11T12:29:41.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.011+0000 7f4619ffb700 1 -- 172.21.15.47:0/548252945 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f462403f050 con 0x55708a148a60 2024-05-11T12:29:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.015+0000 7f4643665b80 10 client.25151 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:41.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.015+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55708a1cc9f0 con 0x55708a148a60 2024-05-11T12:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.018+0000 7f4619ffb700 1 -- 172.21.15.47:0/548252945 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4624010810 con 0x55708a148a60 2024-05-11T12:29:41.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.018+0000 7f4619ffb700 1 -- 172.21.15.47:0/548252945 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4624038740 con 0x55708a148a60 2024-05-11T12:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.019+0000 7f4643665b80 10 client.25151 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.019+0000 7f4643665b80 10 client.25151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:41.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.019+0000 7f4643665b80 20 client.25151 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.019+0000 7f4643665b80 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a1cd110 0x55708a1cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.019+0000 7f4643665b80 4 client.25151 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:29:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.019+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55708a0a8240 con 0x55708a1cd110 2024-05-11T12:29:41.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.020+0000 7f463259c700 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a1cd110 0x55708a1cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:41.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.023+0000 7f4619ffb700 10 client.25151 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:41.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.023+0000 7f463259c700 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a1cd110 0x55708a1cf530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a0055f23049613 server_cookie=2a8e5f9428110bbb in_seq=0 out_seq=0 2024-05-11T12:29:41.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.027+0000 7f4619ffb700 1 -- 172.21.15.47:0/548252945 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55708a0a8240 con 0x55708a1cd110 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.027+0000 7f4619ffb700 10 client.25151 handle_command_reply: tid=0 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.027+0000 7f4643665b80 1 client.25151 shutdown 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 msgr2=0x55708a15a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15a680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55708a148000 tx=0x7f4624027ad0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 shutdown_connections 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55708a1cd110 0x55708a1cf530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55708a14dad0 0x55708a156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55708a1480a0 0x55708a15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 --2- 172.21.15.47:0/548252945 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55708a148a60 0x55708a15a680 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.028+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 >> 172.21.15.47:0/548252945 conn(0x55708a074e60 msgr2=0x55708a13d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:41.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.032+0000 7f4619ffb700 0 client.25151 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.032+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 shutdown_connections 2024-05-11T12:29:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.033+0000 7f4643665b80 1 -- 172.21.15.47:0/548252945 wait complete. 2024-05-11T12:29:41.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.033+0000 7f4643665b80 20 client.25151 trim_cache size 0 max 16384 2024-05-11T12:29:41.035 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:29:41.035 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:29:41.035 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:29:41.035 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:29:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f46200b2a00 msgr2=0x7f46200b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:41.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f46200b2a00 0x7f46200b4ec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4634117710 tx=0x7f462400a300 comp rx=0 tx=0).stop 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f463410aa90 msgr2=0x7f46341161c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f463410aa90 0x7f46341161c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c00ba00 comp rx=0 tx=0).stop 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 shutdown_connections 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f46200b2a00 0x7f46200b4ec0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f463411cd30 0x7f463411a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f463410aa90 0x7f46341161c0 secure :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f461c000c00 tx=0x7f461c00ba00 comp rx=0 tx=0).stop 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 --2- 172.21.15.47:0/4119802024 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4634111d30 0x7f4634116700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:41.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.037+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 >> 172.21.15.47:0/4119802024 conn(0x7f463406e6c0 msgr2=0x7f463411e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:41.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.044+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 shutdown_connections 2024-05-11T12:29:41.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:41.044+0000 7f4638dcf700 1 -- 172.21.15.47:0/4119802024 wait complete. 2024-05-11T12:29:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:41 smithi064 ceph-mon[34537]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 742 MiB/s wr, 1.67k op/s 2024-05-11T12:29:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:41 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:29:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:41 smithi050 ceph-mon[34072]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 742 MiB/s wr, 1.67k op/s 2024-05-11T12:29:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:41 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:29:42.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:41 smithi047 ceph-mon[30375]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 742 MiB/s wr, 1.67k op/s 2024-05-11T12:29:42.002 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:41 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:29:42.750 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:29:42.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:29:43.862 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:29:43.863 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:43 smithi064 ceph-mon[34537]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 827 MiB/s wr, 1.78k op/s 2024-05-11T12:29:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:43 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:43 smithi050 ceph-mon[34072]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 827 MiB/s wr, 1.78k op/s 2024-05-11T12:29:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:43 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:43 smithi047 ceph-mon[30375]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 827 MiB/s wr, 1.78k op/s 2024-05-11T12:29:44.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.035 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:43 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:29:44.832 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:44 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:29:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:44 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:29:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi050 to 177.2M 2024-05-11T12:29:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: Unable to set osd_memory_target on smithi050 to 185895628: error parsing value: Value '185895628' is below minimum 939524096 2024-05-11T12:29:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:45.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:29:45.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:29:45.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:29:45.115 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:44 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:29:45.681 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:45 smithi047 ceph-mon[30375]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 739 MiB/s wr, 1.61k op/s 2024-05-11T12:29:45.681 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:45 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-05-11T12:29:45.681 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:45 smithi047 ceph-mon[30375]: Unable to set osd_memory_target on smithi047 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-11T12:29:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:45 smithi050 ceph-mon[34072]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 739 MiB/s wr, 1.61k op/s 2024-05-11T12:29:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:45 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-05-11T12:29:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:45 smithi050 ceph-mon[34072]: Unable to set osd_memory_target on smithi047 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-11T12:29:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:45 smithi064 ceph-mon[34537]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 739 MiB/s wr, 1.61k op/s 2024-05-11T12:29:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:45 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi047 to 177.2M 2024-05-11T12:29:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:45 smithi064 ceph-mon[34537]: Unable to set osd_memory_target on smithi047 to 185878425: error parsing value: Value '185878425' is below minimum 939524096 2024-05-11T12:29:46.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:46 smithi050 ceph-mon[34072]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 778 MiB/s wr, 1.69k op/s 2024-05-11T12:29:47.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:46 smithi047 ceph-mon[30375]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 778 MiB/s wr, 1.69k op/s 2024-05-11T12:29:47.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:46 smithi064 ceph-mon[34537]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 778 MiB/s wr, 1.69k op/s 2024-05-11T12:29:49.102 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:49 smithi047 ceph-mon[30375]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 844 MiB/s wr, 1.69k op/s 2024-05-11T12:29:49.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:49 smithi064 ceph-mon[34537]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 844 MiB/s wr, 1.69k op/s 2024-05-11T12:29:49.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.375+0000 7f0efd36b700 1 -- 172.21.15.47:0/890523903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef811cd30 msgr2=0x7f0ef811f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:49.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.375+0000 7f0efd36b700 1 --2- 172.21.15.47:0/890523903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef811cd30 0x7f0ef811f120 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0eec009990 tx=0x7f0eec009ca0 comp rx=0 tx=0).stop 2024-05-11T12:29:49.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.383+0000 7f0efd36b700 1 -- 172.21.15.47:0/890523903 shutdown_connections 2024-05-11T12:29:49.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.383+0000 7f0efd36b700 1 --2- 172.21.15.47:0/890523903 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef811cd30 0x7f0ef811f120 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:49.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.383+0000 7f0efd36b700 1 --2- 172.21.15.47:0/890523903 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ef8111c40 0x7f0ef810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:49.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.383+0000 7f0efd36b700 1 --2- 172.21.15.47:0/890523903 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef81120b0 0x7f0ef810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:49.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.383+0000 7f0efd36b700 1 -- 172.21.15.47:0/890523903 >> 172.21.15.47:0/890523903 conn(0x7f0ef806e530 msgr2=0x7f0ef806e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:49.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.388+0000 7f0efd36b700 1 -- 172.21.15.47:0/890523903 shutdown_connections 2024-05-11T12:29:49.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.390+0000 7f0efd36b700 1 -- 172.21.15.47:0/890523903 wait complete. 2024-05-11T12:29:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.393+0000 7f0efd36b700 1 Processor -- start 2024-05-11T12:29:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.393+0000 7f0efd36b700 1 -- start start 2024-05-11T12:29:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.394+0000 7f0efd36b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8111c40 0x7f0ef811b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.394+0000 7f0efd36b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef81120b0 0x7f0ef811b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.394+0000 7f0efd36b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ef811cd30 0x7f0ef81144d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:49.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.394+0000 7f0efd36b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef81224b0 con 0x7f0ef81120b0 2024-05-11T12:29:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.394+0000 7f0efd36b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8122330 con 0x7f0ef8111c40 2024-05-11T12:29:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.394+0000 7f0efd36b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8122630 con 0x7f0ef811cd30 2024-05-11T12:29:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.395+0000 7f0efcb6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ef811cd30 0x7f0ef81144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.395+0000 7f0efcb6a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ef811cd30 0x7f0ef81144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:55582/0 (socket says 172.21.15.47:55582) 2024-05-11T12:29:49.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.395+0000 7f0efcb6a700 1 -- 172.21.15.47:0/2575727664 learned_addr learned my addr 172.21.15.47:0/2575727664 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.396+0000 7f0ef7fff700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8111c40 0x7f0ef811b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.396+0000 7f0ef77fe700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef81120b0 0x7f0ef811b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.396+0000 7f0ef7fff700 1 -- 172.21.15.47:0/2575727664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ef811cd30 msgr2=0x7f0ef81144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.396+0000 7f0ef7fff700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ef811cd30 0x7f0ef81144d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.396+0000 7f0ef7fff700 1 -- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef81120b0 msgr2=0x7f0ef811b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.396+0000 7f0ef7fff700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef81120b0 0x7f0ef811b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:49.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.397+0000 7f0ef7fff700 1 -- 172.21.15.47:0/2575727664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef8114dd0 con 0x7f0ef8111c40 2024-05-11T12:29:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.397+0000 7f0ef7fff700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8111c40 0x7f0ef811b240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000ada0 tx=0x7f0ef000c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.398+0000 7f0ef57fa700 1 -- 172.21.15.47:0/2575727664 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0017020 con 0x7f0ef8111c40 2024-05-11T12:29:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.398+0000 7f0ef57fa700 1 -- 172.21.15.47:0/2575727664 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ef000ce30 con 0x7f0ef8111c40 2024-05-11T12:29:49.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.398+0000 7f0ef57fa700 1 -- 172.21.15.47:0/2575727664 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef0005350 con 0x7f0ef8111c40 2024-05-11T12:29:49.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.399+0000 7f0efd36b700 1 -- 172.21.15.47:0/2575727664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef81150c0 con 0x7f0ef8111c40 2024-05-11T12:29:49.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.399+0000 7f0efd36b700 1 -- 172.21.15.47:0/2575727664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef8119000 con 0x7f0ef8111c40 2024-05-11T12:29:49.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.407+0000 7f0ef57fa700 1 -- 172.21.15.47:0/2575727664 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0ef0004ad0 con 0x7f0ef8111c40 2024-05-11T12:29:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.409+0000 7f0ef57fa700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0ee00b29c0 0x7f0ee00b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.409+0000 7f0ef57fa700 1 -- 172.21.15.47:0/2575727664 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0ef00e18a0 con 0x7f0ef8111c40 2024-05-11T12:29:49.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.407+0000 7f0efd36b700 1 -- 172.21.15.47:0/2575727664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee4005330 con 0x7f0ef8111c40 2024-05-11T12:29:49.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.411+0000 7f0ef77fe700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0ee00b29c0 0x7f0ee00b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:49.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.418+0000 7f0ef77fe700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0ee00b29c0 0x7f0ee00b4e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8115a90 tx=0x7f0ee8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:49.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:49.421+0000 7f0ef57fa700 1 -- 172.21.15.47:0/2575727664 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0ef0021020 con 0x7f0ef8111c40 2024-05-11T12:29:49.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:49 smithi050 ceph-mon[34072]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 844 MiB/s wr, 1.69k op/s 2024-05-11T12:29:50.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.059+0000 7f0efd36b700 1 -- 172.21.15.47:0/2575727664 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ee4005620 con 0x7f0ef8111c40 2024-05-11T12:29:50.062 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:29:50.062 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:29:50.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.061+0000 7f0ef57fa700 1 -- 172.21.15.47:0/2575727664 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f0ef00a9520 con 0x7f0ef8111c40 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 -- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0ee00b29c0 msgr2=0x7f0ee00b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0ee00b29c0 0x7f0ee00b4e80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8115a90 tx=0x7f0ee8005c70 comp rx=0 tx=0).stop 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 -- 172.21.15.47:0/2575727664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8111c40 msgr2=0x7f0ef811b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8111c40 0x7f0ef811b240 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000ada0 tx=0x7f0ef000c330 comp rx=0 tx=0).stop 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 -- 172.21.15.47:0/2575727664 shutdown_connections 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0ef811cd30 0x7f0ef81144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0ee00b29c0 0x7f0ee00b4e80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ef8111c40 0x7f0ef811b240 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0ef000ada0 tx=0x7f0ef000c330 comp rx=0 tx=0).stop 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 --2- 172.21.15.47:0/2575727664 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0ef81120b0 0x7f0ef811b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.067+0000 7f0edeffd700 1 -- 172.21.15.47:0/2575727664 >> 172.21.15.47:0/2575727664 conn(0x7f0ef806e530 msgr2=0x7f0ef8071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.068+0000 7f0edeffd700 1 -- 172.21.15.47:0/2575727664 shutdown_connections 2024-05-11T12:29:50.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:50.068+0000 7f0edeffd700 1 -- 172.21.15.47:0/2575727664 wait complete. 2024-05-11T12:29:50.071 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:29:51.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:51 smithi047 ceph-mon[30375]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 743 MiB/s wr, 1.36k op/s 2024-05-11T12:29:51.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:51 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2575727664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:29:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:51 smithi064 ceph-mon[34537]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 743 MiB/s wr, 1.36k op/s 2024-05-11T12:29:51.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:51 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2575727664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:51 smithi050 ceph-mon[34072]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 743 MiB/s wr, 1.36k op/s 2024-05-11T12:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:51 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2575727664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:29:51.738 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:29:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:29:53.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:53 smithi047 ceph-mon[30375]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 859 MiB/s wr, 1.65k op/s 2024-05-11T12:29:53.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:53 smithi064 ceph-mon[34537]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 859 MiB/s wr, 1.65k op/s 2024-05-11T12:29:53.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:53 smithi050 ceph-mon[34072]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 859 MiB/s wr, 1.65k op/s 2024-05-11T12:29:55.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:55 smithi047 ceph-mon[30375]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 755 MiB/s wr, 1.52k op/s 2024-05-11T12:29:55.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:55 smithi064 ceph-mon[34537]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 755 MiB/s wr, 1.52k op/s 2024-05-11T12:29:55.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:55 smithi050 ceph-mon[34072]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 755 MiB/s wr, 1.52k op/s 2024-05-11T12:29:57.357 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:57 smithi047 ceph-mon[30375]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.54k op/s 2024-05-11T12:29:57.363 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:57 smithi064 ceph-mon[34537]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.54k op/s 2024-05-11T12:29:57.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:57 smithi050 ceph-mon[34072]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.54k op/s 2024-05-11T12:29:58.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.107+0000 7ff943567700 1 -- 172.21.15.47:0/524189586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c111d30 msgr2=0x7ff93c10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.107+0000 7ff943567700 1 --2- 172.21.15.47:0/524189586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c111d30 0x7ff93c10a550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff93401c0d0 tx=0x7ff93401c3e0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.108+0000 7ff943567700 1 -- 172.21.15.47:0/524189586 shutdown_connections 2024-05-11T12:29:58.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.108+0000 7ff943567700 1 --2- 172.21.15.47:0/524189586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff93c11cd30 0x7ff93c11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.108+0000 7ff943567700 1 --2- 172.21.15.47:0/524189586 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c111d30 0x7ff93c10a550 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff93401c0d0 tx=0x7ff93401c3e0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.108+0000 7ff943567700 1 --2- 172.21.15.47:0/524189586 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff93c10aa90 0x7ff93c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.108+0000 7ff943567700 1 -- 172.21.15.47:0/524189586 >> 172.21.15.47:0/524189586 conn(0x7ff93c06e6c0 msgr2=0x7ff93c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.114+0000 7ff943567700 1 -- 172.21.15.47:0/524189586 shutdown_connections 2024-05-11T12:29:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.114+0000 7ff943567700 1 -- 172.21.15.47:0/524189586 wait complete. 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 Processor -- start 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 -- start start 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff93c10aa90 0x7ff93c11c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c11cd30 0x7ff93c118610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff93c118b50 0x7ff93c119000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c114df0 con 0x7ff93c118b50 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c114c70 con 0x7ff93c11cd30 2024-05-11T12:29:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.115+0000 7ff943567700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c114f70 con 0x7ff93c10aa90 2024-05-11T12:29:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.124+0000 7ff940b02700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c11cd30 0x7ff93c118610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.124+0000 7ff940b02700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c11cd30 0x7ff93c118610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:43214/0 (socket says 172.21.15.47:43214) 2024-05-11T12:29:58.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.124+0000 7ff940b02700 1 -- 172.21.15.47:0/949910560 learned_addr learned my addr 172.21.15.47:0/949910560 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.125+0000 7ff941b04700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff93c118b50 0x7ff93c119000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.129+0000 7ff941303700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff93c10aa90 0x7ff93c11c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.130+0000 7ff941b04700 1 -- 172.21.15.47:0/949910560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff93c10aa90 msgr2=0x7ff93c11c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.130+0000 7ff941b04700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff93c10aa90 0x7ff93c11c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.130+0000 7ff941b04700 1 -- 172.21.15.47:0/949910560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c11cd30 msgr2=0x7ff93c118610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.130+0000 7ff941b04700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c11cd30 0x7ff93c118610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.130+0000 7ff941b04700 1 -- 172.21.15.47:0/949910560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c07b830 con 0x7ff93c118b50 2024-05-11T12:29:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.135+0000 7ff941b04700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff93c118b50 0x7ff93c119000 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b3c0 tx=0x7ff93800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.137+0000 7ff9327fc700 1 -- 172.21.15.47:0/949910560 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938015020 con 0x7ff93c118b50 2024-05-11T12:29:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.137+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c07bb20 con 0x7ff93c118b50 2024-05-11T12:29:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.137+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c07c060 con 0x7ff93c118b50 2024-05-11T12:29:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.137+0000 7ff9327fc700 1 -- 172.21.15.47:0/949910560 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff938011ce0 con 0x7ff93c118b50 2024-05-11T12:29:58.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.137+0000 7ff9327fc700 1 -- 172.21.15.47:0/949910560 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938015020 con 0x7ff93c118b50 2024-05-11T12:29:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.158+0000 7ff94ddfdb80 1 Processor -- start 2024-05-11T12:29:58.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.158+0000 7ff9327fc700 1 -- 172.21.15.47:0/949910560 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff938018020 con 0x7ff93c118b50 2024-05-11T12:29:58.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.158+0000 7ff94ddfdb80 1 -- start start 2024-05-11T12:29:58.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.164+0000 7ff9327fc700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff9280b2cb0 0x7ff9280b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.165+0000 7ff9327fc700 1 -- 172.21.15.47:0/949910560 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff938011630 con 0x7ff93c118b50 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.166+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.167+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.167+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.167+0000 7ff94ddfdb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56233b012860 con 0x56233b0c00a0 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.167+0000 7ff94ddfdb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56233afb17c0 con 0x56233b0c5ad0 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.167+0000 7ff94ddfdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56233b00e820 con 0x56233b0c0a60 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.168+0000 7ff940b02700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.168+0000 7ff940b02700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59990/0 (socket says 172.21.15.47:59990) 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.168+0000 7ff940b02700 1 -- 172.21.15.47:0/4293649372 learned_addr learned my addr 172.21.15.47:0/4293649372 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:58.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.169+0000 7ff941b04700 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.170+0000 7ff940b02700 1 -- 172.21.15.47:0/4293649372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 msgr2=0x56233b0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.170+0000 7ff940b02700 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.170+0000 7ff940b02700 1 -- 172.21.15.47:0/4293649372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 msgr2=0x56233b0c5eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.170+0000 7ff940b02700 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0c5eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.170+0000 7ff940b02700 1 -- 172.21.15.47:0/4293649372 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233af5f8d0 con 0x56233b0c00a0 2024-05-11T12:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.171+0000 7ff940b02700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff9280b2cb0 0x7ff9280b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.174+0000 7ff941b04700 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:29:58.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.176+0000 7ff940b02700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff9280b2cb0 0x7ff9280b5170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009fd0 tx=0x7ff92c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:58.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.179+0000 7ff940b02700 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c0520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009c50 tx=0x7ff92c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b51865a5d6b076 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:58.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.184+0000 7ff9277fe700 1 -- 172.21.15.47:0/4293649372 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c010e70 con 0x56233b0c00a0 2024-05-11T12:29:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.184+0000 7ff9277fe700 1 -- 172.21.15.47:0/4293649372 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff92c023070 con 0x56233b0c00a0 2024-05-11T12:29:58.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.184+0000 7ff9277fe700 1 -- 172.21.15.47:0/4293649372 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c01e3f0 con 0x56233b0c00a0 2024-05-11T12:29:58.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.187+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4293649372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 msgr2=0x56233b0c0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.187+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c0520 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009c50 tx=0x7ff92c009f60 comp rx=0 tx=0).stop 2024-05-11T12:29:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.191+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4293649372 shutdown_connections 2024-05-11T12:29:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.191+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.191+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.191+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4293649372 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c0520 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.191+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4293649372 >> 172.21.15.47:0/4293649372 conn(0x56233afece60 msgr2=0x56233b0ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.195+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4293649372 shutdown_connections 2024-05-11T12:29:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.195+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4293649372 wait complete. 2024-05-11T12:29:58.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.196+0000 7ff94ddfdb80 1 -- start start 2024-05-11T12:29:58.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.196+0000 7ff94ddfdb80 1 Processor -- start 2024-05-11T12:29:58.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.197+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.197+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.198+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.198+0000 7ff94ddfdb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56233b0cce90 con 0x56233b0c00a0 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.198+0000 7ff94ddfdb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56233af78c30 con 0x56233b0c5ad0 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.198+0000 7ff94ddfdb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56233b0cd010 con 0x56233b0c0a60 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.198+0000 7ff94ddfdb80 10 client.0 _ll_register_callbacks cb 0x56233afaa920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.200+0000 7ff941303700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.201+0000 7ff941303700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59994/0 (socket says 172.21.15.47:59994) 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.201+0000 7ff941303700 1 -- 172.21.15.47:0/215050717 learned_addr learned my addr 172.21.15.47:0/215050717 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.201+0000 7ff941b04700 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.201+0000 7ff940b02700 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.202+0000 7ff941303700 1 -- 172.21.15.47:0/215050717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 msgr2=0x56233b0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:58.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.202+0000 7ff941303700 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.202+0000 7ff941303700 1 -- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 msgr2=0x56233b0c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.202+0000 7ff941303700 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.202+0000 7ff941303700 1 -- 172.21.15.47:0/215050717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233afae690 con 0x56233b0c00a0 2024-05-11T12:29:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.204+0000 7ff941303700 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0cc530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff93c06f410 tx=0x7ff93401cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95436794a37f5313 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:58.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.204+0000 7ff924ff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.206+0000 7ff924ff9700 1 -- 172.21.15.47:0/215050717 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff934005a40 con 0x56233b0c00a0 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.206+0000 7ff924ff9700 1 -- 172.21.15.47:0/215050717 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff934003a00 con 0x56233b0c00a0 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.206+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92c023070 con 0x56233b0c00a0 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.207+0000 7ff924ff9700 1 -- 172.21.15.47:0/215050717 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff934003c00 con 0x56233b0c00a0 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.207+0000 7ff94ddfdb80 10 client.15579 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.207+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56233b0d76f0 con 0x56233b0c00a0 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.208+0000 7ff924ff9700 1 -- 172.21.15.47:0/215050717 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff92c023070 con 0x56233b0c00a0 2024-05-11T12:29:58.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.208+0000 7ff924ff9700 1 -- 172.21.15.47:0/215050717 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff934022070 con 0x56233b0c00a0 2024-05-11T12:29:58.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.208+0000 7ff94ddfdb80 10 client.15579 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:58.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.208+0000 7ff94ddfdb80 10 client.15579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:58.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.208+0000 7ff94ddfdb80 20 client.15579 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:58.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.208+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b0d9cd0 0x56233b0dc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.209+0000 7ff94ddfdb80 4 client.15579 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:29:58.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.209+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56233b020240 con 0x56233b0d9cd0 2024-05-11T12:29:58.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.210+0000 7ff940b02700 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b0d9cd0 0x56233b0dc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.213+0000 7ff940b02700 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b0d9cd0 0x56233b0dc0c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959c3365ad76727e server_cookie=4e91f9fe6920d642 in_seq=0 out_seq=0 2024-05-11T12:29:58.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.213+0000 7ff924ff9700 10 client.15579 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:58.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.215+0000 7ff924ff9700 1 -- 172.21.15.47:0/215050717 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56233b020240 con 0x56233b0d9cd0 2024-05-11T12:29:58.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.216+0000 7ff924ff9700 10 client.15579 handle_command_reply: tid=0 2024-05-11T12:29:58.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.216+0000 7ff94ddfdb80 1 client.15579 shutdown 2024-05-11T12:29:58.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.216+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 msgr2=0x56233b0cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.216+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0cc530 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff93c06f410 tx=0x7ff93401cf90 comp rx=0 tx=0).stop 2024-05-11T12:29:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.219+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 shutdown_connections 2024-05-11T12:29:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.219+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b0d9cd0 0x56233b0dc0c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.219+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.219+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0cc530 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.219+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/215050717 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.219+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 >> 172.21.15.47:0/215050717 conn(0x56233afece60 msgr2=0x56233b0b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.221+0000 7ff924ff9700 0 client.15579 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.223+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 shutdown_connections 2024-05-11T12:29:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.223+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/215050717 wait complete. 2024-05-11T12:29:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.223+0000 7ff94ddfdb80 20 client.15579 trim_cache size 0 max 16384 2024-05-11T12:29:58.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.258+0000 7ff94ddfdb80 1 Processor -- start 2024-05-11T12:29:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.259+0000 7ff94ddfdb80 1 -- start start 2024-05-11T12:29:58.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.260+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.262+0000 7ff941303700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.262+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0d5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.263+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0e1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.263+0000 7ff940b02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0d5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.263+0000 7ff940b02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0d5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40814/0 (socket says 172.21.15.47:40814) 2024-05-11T12:29:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.263+0000 7ff940b02700 1 -- 172.21.15.47:0/4285087514 learned_addr learned my addr 172.21.15.47:0/4285087514 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:58.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.263+0000 7ff941b04700 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.263+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000b820 con 0x56233b0c00a0 2024-05-11T12:29:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.264+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000e0a0 con 0x56233b0c5ad0 2024-05-11T12:29:58.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.265+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000be90 con 0x56233b0c0a60 2024-05-11T12:29:58.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.266+0000 7ff940b02700 1 -- 172.21.15.47:0/4285087514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 msgr2=0x56233b0e1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.266+0000 7ff940b02700 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0e1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.266+0000 7ff940b02700 1 -- 172.21.15.47:0/4285087514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 msgr2=0x56233b0c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.267+0000 7ff940b02700 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.267+0000 7ff940b02700 1 -- 172.21.15.47:0/4285087514 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56233b0c7db0 con 0x56233b0c0a60 2024-05-11T12:29:58.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.269+0000 7ff940b02700 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0d5f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff92c016ad0 tx=0x7ff92c010040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=450b2fb32ea5b029 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.273+0000 7ff926ffd700 1 -- 172.21.15.47:0/4285087514 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c034070 con 0x56233b0c0a60 2024-05-11T12:29:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.274+0000 7ff926ffd700 1 -- 172.21.15.47:0/4285087514 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff92c023070 con 0x56233b0c0a60 2024-05-11T12:29:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.274+0000 7ff926ffd700 1 -- 172.21.15.47:0/4285087514 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c004550 con 0x56233b0c0a60 2024-05-11T12:29:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.275+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 msgr2=0x56233b0d5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.275+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0d5f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff92c016ad0 tx=0x7ff92c010040 comp rx=0 tx=0).stop 2024-05-11T12:29:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.276+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 shutdown_connections 2024-05-11T12:29:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.276+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c5ad0 0x56233b0e1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.276+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c00a0 0x56233b0c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.276+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/4285087514 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0d5f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.276+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 >> 172.21.15.47:0/4285087514 conn(0x56233afece60 msgr2=0x56233b0bf920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.278+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 shutdown_connections 2024-05-11T12:29:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.279+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/4285087514 wait complete. 2024-05-11T12:29:58.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.279+0000 7ff94ddfdb80 1 -- start start 2024-05-11T12:29:58.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.279+0000 7ff94ddfdb80 1 Processor -- start 2024-05-11T12:29:58.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.283+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c00a0 0x56233b0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff94ddfdb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c5ad0 0x56233b0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff941303700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c00a0 0x56233b0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff941303700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c00a0 0x56233b0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:43238/0 (socket says 172.21.15.47:43238) 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff940b02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff940b02700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40822/0 (socket says 172.21.15.47:40822) 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff941303700 1 -- 172.21.15.47:0/3446800114 learned_addr learned my addr 172.21.15.47:0/3446800114 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000c560 con 0x56233b0c5ad0 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000d9d0 con 0x56233b0c00a0 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.284+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff92000cc30 con 0x56233b0c0a60 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff94ddfdb80 10 client.0 _ll_register_callbacks cb 0x56233afaa920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff940b02700 1 -- 172.21.15.47:0/3446800114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c00a0 msgr2=0x56233b0d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff940b02700 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c00a0 0x56233b0d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff940b02700 1 -- 172.21.15.47:0/3446800114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c5ad0 msgr2=0x56233b0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:29:58.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff940b02700 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c5ad0 0x56233b0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff940b02700 1 -- 172.21.15.47:0/3446800114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56233b0ceaf0 con 0x56233b0c0a60 2024-05-11T12:29:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff940b02700 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cdcc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x56233b0b7650 tx=0x7ff92c014f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0b98c598878276e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:29:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.285+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff92c023070 con 0x56233b0c0a60 2024-05-11T12:29:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.286+0000 7ff924ff9700 10 client.25222 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:29:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.286+0000 7ff924ff9700 1 -- 172.21.15.47:0/3446800114 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c034070 con 0x56233b0c0a60 2024-05-11T12:29:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.286+0000 7ff924ff9700 1 -- 172.21.15.47:0/3446800114 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff92c014470 con 0x56233b0c0a60 2024-05-11T12:29:58.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.286+0000 7ff924ff9700 1 -- 172.21.15.47:0/3446800114 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff92c040050 con 0x56233b0c0a60 2024-05-11T12:29:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.288+0000 7ff94ddfdb80 10 client.25222 fetch_fsmap learned FSMap version 20 2024-05-11T12:29:58.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.288+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({fsmap=20}) -- 0x56233b1444f0 con 0x56233b0c0a60 2024-05-11T12:29:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.292+0000 7ff924ff9700 1 -- 172.21.15.47:0/3446800114 <== mon.1 v2:172.21.15.64:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff92c01d030 con 0x56233b0c0a60 2024-05-11T12:29:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.292+0000 7ff94ddfdb80 10 client.25222 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:29:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.292+0000 7ff94ddfdb80 10 client.25222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:29:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.292+0000 7ff94ddfdb80 20 client.25222 populate_metadata read hostname 'smithi047' 2024-05-11T12:29:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.292+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b1446c0 0x56233b146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:29:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.292+0000 7ff94ddfdb80 4 client.25222 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:29:58.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.292+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff92c023070 con 0x56233b1446c0 2024-05-11T12:29:58.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.293+0000 7ff941303700 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b1446c0 0x56233b146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:29:58.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.296+0000 7ff941303700 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b1446c0 0x56233b146ab0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd5e96e9649a758 server_cookie=4f6efab7c061b79e in_seq=0 out_seq=0 2024-05-11T12:29:58.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.297+0000 7ff924ff9700 10 client.25222 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:58.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.298+0000 7ff924ff9700 1 -- 172.21.15.47:0/3446800114 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff92c023070 con 0x56233b1446c0 2024-05-11T12:29:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.298+0000 7ff924ff9700 10 client.25222 handle_command_reply: tid=0 2024-05-11T12:29:58.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.298+0000 7ff94ddfdb80 1 client.25222 shutdown 2024-05-11T12:29:58.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.299+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 msgr2=0x56233b0cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.299+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cdcc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x56233b0b7650 tx=0x7ff92c014f70 comp rx=0 tx=0).stop 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 shutdown_connections 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56233b1446c0 0x56233b146ab0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56233b0c5ad0 0x56233b0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56233b0c00a0 0x56233b0d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 --2- 172.21.15.47:0/3446800114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56233b0c0a60 0x56233b0cdcc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 >> 172.21.15.47:0/3446800114 conn(0x56233afece60 msgr2=0x56233b0c34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff924ff9700 0 client.25222 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 shutdown_connections 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 1 -- 172.21.15.47:0/3446800114 wait complete. 2024-05-11T12:29:58.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.300+0000 7ff94ddfdb80 20 client.25222 trim_cache size 0 max 16384 2024-05-11T12:29:58.307 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:29:58.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.307+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff9280b2cb0 msgr2=0x7ff9280b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.307+0000 7ff943567700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff9280b2cb0 0x7ff9280b5170 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff92c009fd0 tx=0x7ff92c008040 comp rx=0 tx=0).stop 2024-05-11T12:29:58.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.307+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff93c118b50 msgr2=0x7ff93c119000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.307+0000 7ff943567700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff93c118b50 0x7ff93c119000 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b3c0 tx=0x7ff93800b6d0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.308+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 shutdown_connections 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.308+0000 7ff943567700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff93c118b50 0x7ff93c119000 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.308+0000 7ff943567700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff93c10aa90 0x7ff93c11c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.308+0000 7ff943567700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff9280b2cb0 0x7ff9280b5170 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.308+0000 7ff943567700 1 --2- 172.21.15.47:0/949910560 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff93c11cd30 0x7ff93c118610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.308+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 >> 172.21.15.47:0/949910560 conn(0x7ff93c06e6c0 msgr2=0x7ff93c11e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.309+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 shutdown_connections 2024-05-11T12:29:58.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:29:58.310+0000 7ff943567700 1 -- 172.21.15.47:0/949910560 wait complete. 2024-05-11T12:29:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:29:59 smithi064 ceph-mon[34537]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 862 MiB/s wr, 1.52k op/s 2024-05-11T12:29:59.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:29:59 smithi050 ceph-mon[34072]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 862 MiB/s wr, 1.52k op/s 2024-05-11T12:29:59.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:29:59 smithi047 ceph-mon[30375]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 862 MiB/s wr, 1.52k op/s 2024-05-11T12:30:00.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:00 smithi050 ceph-mon[34072]: overall HEALTH_OK 2024-05-11T12:30:00.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:00 smithi047 ceph-mon[30375]: overall HEALTH_OK 2024-05-11T12:30:00.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:00 smithi064 ceph-mon[34537]: overall HEALTH_OK 2024-05-11T12:30:01.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:30:01.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:30:01.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:01 smithi050 ceph-mon[34072]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 747 MiB/s wr, 1.26k op/s 2024-05-11T12:30:01.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:01 smithi047 ceph-mon[30375]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 747 MiB/s wr, 1.26k op/s 2024-05-11T12:30:01.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:01 smithi064 ceph-mon[34537]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 747 MiB/s wr, 1.26k op/s 2024-05-11T12:30:01.867 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:30:03.416 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:03 smithi064 ceph-mon[34537]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 919 MiB/s wr, 1.72k op/s 2024-05-11T12:30:03.435 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:03 smithi047 ceph-mon[30375]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 919 MiB/s wr, 1.72k op/s 2024-05-11T12:30:03.448 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:03 smithi050 ceph-mon[34072]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 919 MiB/s wr, 1.72k op/s 2024-05-11T12:30:05.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:05 smithi064 ceph-mon[34537]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 758 MiB/s wr, 1.49k op/s 2024-05-11T12:30:05.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:05 smithi050 ceph-mon[34072]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 758 MiB/s wr, 1.49k op/s 2024-05-11T12:30:05.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:05 smithi047 ceph-mon[30375]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 758 MiB/s wr, 1.49k op/s 2024-05-11T12:30:07.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.024+0000 7f4fa8834700 1 -- 172.21.15.47:0/2570063221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fa411cd30 msgr2=0x7f4fa411f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.024+0000 7f4fa8834700 1 --2- 172.21.15.47:0/2570063221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fa411cd30 0x7f4fa411f120 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009990 tx=0x7f4f98009ca0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.025+0000 7f4fa8834700 1 -- 172.21.15.47:0/2570063221 shutdown_connections 2024-05-11T12:30:07.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.025+0000 7f4fa8834700 1 --2- 172.21.15.47:0/2570063221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fa411cd30 0x7f4fa411f120 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.025+0000 7f4fa8834700 1 --2- 172.21.15.47:0/2570063221 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fa4111d30 0x7f4fa410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.025+0000 7f4fa8834700 1 --2- 172.21.15.47:0/2570063221 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa410aa90 0x7f4fa410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.025+0000 7f4fa8834700 1 -- 172.21.15.47:0/2570063221 >> 172.21.15.47:0/2570063221 conn(0x7f4fa406e6c0 msgr2=0x7f4fa406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:07.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.029+0000 7f4fa8834700 1 -- 172.21.15.47:0/2570063221 shutdown_connections 2024-05-11T12:30:07.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.029+0000 7f4fa8834700 1 -- 172.21.15.47:0/2570063221 wait complete. 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.031+0000 7f4fa8834700 1 Processor -- start 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.031+0000 7f4fa8834700 1 -- start start 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.032+0000 7f4fa8834700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fa410aa90 0x7f4fa41ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.032+0000 7f4fa8834700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fa4111d30 0x7f4fa41aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.032+0000 7f4fa8834700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa411cd30 0x7f4fa41a8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.032+0000 7f4fa8834700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4122700 con 0x7f4fa410aa90 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.032+0000 7f4fa8834700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4122580 con 0x7f4fa4111d30 2024-05-11T12:30:07.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.033+0000 7f4fa8834700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4122880 con 0x7f4fa411cd30 2024-05-11T12:30:07.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.033+0000 7f4fa2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa411cd30 0x7f4fa41a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.033+0000 7f4fa2d9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa411cd30 0x7f4fa41a8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49330/0 (socket says 172.21.15.47:49330) 2024-05-11T12:30:07.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.033+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1881351659 learned_addr learned my addr 172.21.15.47:0/1881351659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.034+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1881351659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fa4111d30 msgr2=0x7f4fa41aebc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:30:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.034+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fa4111d30 0x7f4fa41aebc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.034+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fa410aa90 msgr2=0x7f4fa41ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.034+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fa410aa90 0x7f4fa41ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.034+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1881351659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa41a8fe0 con 0x7f4fa411cd30 2024-05-11T12:30:07.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.035+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa411cd30 0x7f4fa41a8800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f98003ae0 tx=0x7f4f98003bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:07.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.036+0000 7f4f937fe700 1 -- 172.21.15.47:0/1881351659 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98026070 con 0x7f4fa411cd30 2024-05-11T12:30:07.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.036+0000 7f4f937fe700 1 -- 172.21.15.47:0/1881351659 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f98003680 con 0x7f4fa411cd30 2024-05-11T12:30:07.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.037+0000 7f4fa8834700 1 -- 172.21.15.47:0/1881351659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa41a9270 con 0x7f4fa411cd30 2024-05-11T12:30:07.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.037+0000 7f4fa8834700 1 -- 172.21.15.47:0/1881351659 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa41a9690 con 0x7f4fa411cd30 2024-05-11T12:30:07.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.038+0000 7f4f937fe700 1 -- 172.21.15.47:0/1881351659 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f98021410 con 0x7f4fa411cd30 2024-05-11T12:30:07.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.049+0000 7f4f937fe700 1 -- 172.21.15.47:0/1881351659 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4f98029070 con 0x7f4fa411cd30 2024-05-11T12:30:07.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.054+0000 7f4fb30cab80 1 Processor -- start 2024-05-11T12:30:07.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.055+0000 7f4fb30cab80 1 -- start start 2024-05-11T12:30:07.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.055+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.055+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.055+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dbe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.055+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b8f9d0f9c0 con 0x55b8f9dc3060 2024-05-11T12:30:07.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.055+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b8f9c74e20 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.055+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b8f9d0b980 con 0x55b8f9dc3980 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.056+0000 7f4fa1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.056+0000 7f4fa1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38818/0 (socket says 172.21.15.47:38818) 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.056+0000 7f4fa1d9b700 1 -- 172.21.15.47:0/2997886490 learned_addr learned my addr 172.21.15.47:0/2997886490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.056+0000 7f4fa1d9b700 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dbe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.057+0000 7f4fa1d9b700 1 -- 172.21.15.47:0/2997886490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 msgr2=0x55b8f9dbd590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.057+0000 7f4fa1d9b700 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dbd590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.057+0000 7f4fa1d9b700 1 -- 172.21.15.47:0/2997886490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 msgr2=0x55b8f9dbe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.057+0000 7f4fa1d9b700 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dbe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.057+0000 7f4fa1d9b700 1 -- 172.21.15.47:0/2997886490 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f9c5c950 con 0x55b8f9dc3060 2024-05-11T12:30:07.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.057+0000 7f4f937fe700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4f8c0b2960 0x7f4f8c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.058+0000 7f4fa1d9b700 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc3440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009e30 tx=0x7f4f940090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce49a53550d7860e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:07.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.058+0000 7f4f937fe700 1 -- 172.21.15.47:0/1881351659 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4f980e3180 con 0x7f4fa411cd30 2024-05-11T12:30:07.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.059+0000 7f4f8bfff700 1 -- 172.21.15.47:0/2997886490 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9400f040 con 0x55b8f9dc3060 2024-05-11T12:30:07.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.060+0000 7f4f8bfff700 1 -- 172.21.15.47:0/2997886490 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f940044e0 con 0x55b8f9dc3060 2024-05-11T12:30:07.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.061+0000 7f4f8bfff700 1 -- 172.21.15.47:0/2997886490 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94003bb0 con 0x55b8f9dc3060 2024-05-11T12:30:07.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.061+0000 7f4fb30cab80 1 -- 172.21.15.47:0/2997886490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 msgr2=0x55b8f9dc3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.061+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc3440 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4f94009e30 tx=0x7f4f940090f0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.063+0000 7f4fa259c700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4f8c0b2960 0x7f4f8c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.066+0000 7f4fb30cab80 1 -- 172.21.15.47:0/2997886490 shutdown_connections 2024-05-11T12:30:07.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.066+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dbe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.066+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc3440 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.066+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/2997886490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dbd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.066+0000 7f4fb30cab80 1 -- 172.21.15.47:0/2997886490 >> 172.21.15.47:0/2997886490 conn(0x55b8f9ce9bf0 msgr2=0x55b8f9db8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.069+0000 7f4fb30cab80 1 -- 172.21.15.47:0/2997886490 shutdown_connections 2024-05-11T12:30:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.070+0000 7f4fb30cab80 1 -- 172.21.15.47:0/2997886490 wait complete. 2024-05-11T12:30:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.071+0000 7f4fb30cab80 1 -- start start 2024-05-11T12:30:07.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.071+0000 7f4fb30cab80 1 Processor -- start 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.071+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dc9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.071+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.071+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dc6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.071+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55b8f9dca4f0 con 0x55b8f9dc3060 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.072+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55b8f9cc8f50 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.072+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55b8f9dca670 con 0x55b8f9dc3980 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.072+0000 7f4fa259c700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4f8c0b2960 0x7f4f8c0b4e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00abd0 tx=0x7f4f9c009230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:07.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.072+0000 7f4fb30cab80 10 client.0 _ll_register_callbacks cb 0x55b8f9ca79d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.072+0000 7f4fa259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dc9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.072+0000 7f4fa259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dc9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:42292/0 (socket says 172.21.15.47:42292) 2024-05-11T12:30:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.072+0000 7f4fa259c700 1 -- 172.21.15.47:0/291509480 learned_addr learned my addr 172.21.15.47:0/291509480 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.073+0000 7f4fa259c700 1 -- 172.21.15.47:0/291509480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 msgr2=0x55b8f9dc6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:07.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.073+0000 7f4fa259c700 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dc6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.073+0000 7f4fa259c700 1 -- 172.21.15.47:0/291509480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 msgr2=0x55b8f9dc68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.073+0000 7f4fa259c700 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.073+0000 7f4fa259c700 1 -- 172.21.15.47:0/291509480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f9cc67e0 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.074+0000 7f4fa259c700 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dc9550 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00d9b0 tx=0x7f4f9c004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b2592b5b2eff885 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.074+0000 7f4f89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.075+0000 7f4f89ffb700 1 -- 172.21.15.47:0/291509480 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c013070 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.075+0000 7f4f89ffb700 1 -- 172.21.15.47:0/291509480 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4f9c018bc0 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.075+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f940044e0 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.076+0000 7f4f89ffb700 1 -- 172.21.15.47:0/291509480 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c006e60 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.077+0000 7f4fb30cab80 10 client.25175 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:07.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.077+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b8f9dd4a70 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4f89ffb700 1 -- 172.21.15.47:0/291509480 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f9c011070 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4f89ffb700 1 -- 172.21.15.47:0/291509480 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4f9c02d420 con 0x55b8f9dbdbb0 2024-05-11T12:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4fb30cab80 10 client.25175 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:07.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4fb30cab80 10 client.25175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4fb30cab80 20 client.25175 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9dd7000 0x55b8f9dd9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4fb30cab80 4 client.25175 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.080+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f940044e0 con 0x55b8f9dd7000 2024-05-11T12:30:07.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.081+0000 7f4fa1d9b700 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9dd7000 0x55b8f9dd9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.082+0000 7f4fa1d9b700 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9dd7000 0x55b8f9dd9470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211c5e7b168ef4 server_cookie=ac26e132287c6c3a in_seq=0 out_seq=0 2024-05-11T12:30:07.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.084+0000 7f4f89ffb700 10 client.25175 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:07.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.084+0000 7f4f89ffb700 1 -- 172.21.15.47:0/291509480 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f4f940044e0 con 0x55b8f9dd7000 2024-05-11T12:30:07.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.084+0000 7f4f89ffb700 10 client.25175 handle_command_reply: tid=0 2024-05-11T12:30:07.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.084+0000 7f4fb30cab80 1 client.25175 shutdown 2024-05-11T12:30:07.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 msgr2=0x55b8f9dc9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dc9550 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00d9b0 tx=0x7f4f9c004bd0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 shutdown_connections 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9dd7000 0x55b8f9dd9470 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dc6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dbdbb0 0x55b8f9dc9550 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00d9b0 tx=0x7f4f9c004bd0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/291509480 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dc3060 0x55b8f9dc68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 >> 172.21.15.47:0/291509480 conn(0x55b8f9ce9bf0 msgr2=0x55b8f9db2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.093+0000 7f4f89ffb700 0 client.25175 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.096+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 shutdown_connections 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.096+0000 7f4fb30cab80 1 -- 172.21.15.47:0/291509480 wait complete. 2024-05-11T12:30:07.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.096+0000 7f4fb30cab80 20 client.25175 trim_cache size 0 max 16384 2024-05-11T12:30:07.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.157+0000 7f4fb30cab80 1 Processor -- start 2024-05-11T12:30:07.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.159+0000 7f4fb30cab80 1 -- start start 2024-05-11T12:30:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.160+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3060 0x55b8f9dbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.161+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3980 0x55b8f9dc5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.161+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9db27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.162+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa41ba9d0 con 0x55b8f9dd35d0 2024-05-11T12:30:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.162+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa41bd250 con 0x55b8f9dc3980 2024-05-11T12:30:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.162+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa41bb040 con 0x55b8f9dc3060 2024-05-11T12:30:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9db27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9db27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38842/0 (socket says 172.21.15.47:38842) 2024-05-11T12:30:07.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1731496228 learned_addr learned my addr 172.21.15.47:0/1731496228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1731496228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3060 msgr2=0x55b8f9dbd590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:30:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1731496228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3060 0x55b8f9dbd590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1731496228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3980 msgr2=0x55b8f9dc5ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:30:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1731496228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3980 0x55b8f9dc5ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.163+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1731496228 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8f9d9ab70 con 0x55b8f9dd35d0 2024-05-11T12:30:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.164+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1731496228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9db27f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f980aa400 tx=0x7f4f980b1140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b407441641f3f7dc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.164+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/1731496228 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980095d0 con 0x55b8f9dd35d0 2024-05-11T12:30:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.164+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/1731496228 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f980aa210 con 0x55b8f9dd35d0 2024-05-11T12:30:07.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.164+0000 7f4f8b7fe700 1 -- 172.21.15.47:0/1731496228 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980aabc0 con 0x55b8f9dd35d0 2024-05-11T12:30:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.166+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1731496228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 msgr2=0x55b8f9db27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.166+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1731496228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9db27f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4f980aa400 tx=0x7f4f980b1140 comp rx=0 tx=0).stop 2024-05-11T12:30:07.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.170+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1731496228 shutdown_connections 2024-05-11T12:30:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.170+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1731496228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9db27f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.170+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1731496228 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3060 0x55b8f9dbd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.170+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1731496228 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3980 0x55b8f9dc5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.170+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1731496228 >> 172.21.15.47:0/1731496228 conn(0x55b8f9ce9bf0 msgr2=0x55b8f9db20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.171+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1731496228 shutdown_connections 2024-05-11T12:30:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.172+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1731496228 wait complete. 2024-05-11T12:30:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.173+0000 7f4fb30cab80 1 -- start start 2024-05-11T12:30:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.173+0000 7f4fb30cab80 1 Processor -- start 2024-05-11T12:30:07.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.174+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3060 0x55b8f9dcfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.175+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.175+0000 7f4fb30cab80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9dcd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.175+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa41bb710 con 0x55b8f9dd35d0 2024-05-11T12:30:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.176+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa41bcb80 con 0x55b8f9dc3060 2024-05-11T12:30:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.176+0000 7f4fb30cab80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa41bbde0 con 0x55b8f9dc3980 2024-05-11T12:30:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.176+0000 7f4fb30cab80 10 client.0 _ll_register_callbacks cb 0x55b8f9ca79d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:07.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.176+0000 7f4fa2d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9dcd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.177+0000 7f4fa1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.177+0000 7f4fa1d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49358/0 (socket says 172.21.15.47:49358) 2024-05-11T12:30:07.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.177+0000 7f4fa1d9b700 1 -- 172.21.15.47:0/1988067712 learned_addr learned my addr 172.21.15.47:0/1988067712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.178+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1988067712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 msgr2=0x55b8f9dccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.179+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.179+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1988067712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3060 msgr2=0x55b8f9dcfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.179+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3060 0x55b8f9dcfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.179+0000 7f4fa2d9d700 1 -- 172.21.15.47:0/1988067712 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8f9dcdaa0 con 0x55b8f9dd35d0 2024-05-11T12:30:07.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.180+0000 7f4fa2d9d700 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9dcd4e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4f98006010 tx=0x7f4f980aa180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb600a364f1a0e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:07.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.180+0000 7f4f89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:30:07.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.181+0000 7f4f89ffb700 1 -- 172.21.15.47:0/1988067712 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f980c3020 con 0x55b8f9dd35d0 2024-05-11T12:30:07.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.181+0000 7f4f89ffb700 1 -- 172.21.15.47:0/1988067712 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4f980a6280 con 0x55b8f9dd35d0 2024-05-11T12:30:07.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.181+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f980aa210 con 0x55b8f9dd35d0 2024-05-11T12:30:07.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.182+0000 7f4f89ffb700 1 -- 172.21.15.47:0/1988067712 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4f980ca050 con 0x55b8f9dd35d0 2024-05-11T12:30:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.183+0000 7f4fb30cab80 10 client.15594 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.184+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b8f9e436e0 con 0x55b8f9dd35d0 2024-05-11T12:30:07.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.196+0000 7f4f89ffb700 1 -- 172.21.15.47:0/1988067712 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f980095d0 con 0x55b8f9dd35d0 2024-05-11T12:30:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.197+0000 7f4f89ffb700 1 -- 172.21.15.47:0/1988067712 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4f980bfac0 con 0x55b8f9dd35d0 2024-05-11T12:30:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.198+0000 7f4fb30cab80 10 client.15594 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.198+0000 7f4fb30cab80 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.198+0000 7f4fb30cab80 20 client.15594 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.198+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9e43c70 0x55b8f9e46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.198+0000 7f4fb30cab80 4 client.15594 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:30:07.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.198+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f4f980aa210 con 0x55b8f9e43c70 2024-05-11T12:30:07.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.199+0000 7f4fa259c700 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9e43c70 0x55b8f9e46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:07.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.209+0000 7f4fa259c700 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9e43c70 0x55b8f9e46060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ad27757f413f99 server_cookie=4bf35fbafe57b18f in_seq=0 out_seq=0 2024-05-11T12:30:07.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.213+0000 7f4f89ffb700 10 client.15594 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:07.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:07 smithi047 ceph-mon[30375]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 819 MiB/s wr, 1.84k op/s 2024-05-11T12:30:07.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:07 smithi050 ceph-mon[34072]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 819 MiB/s wr, 1.84k op/s 2024-05-11T12:30:07.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:07 smithi064 ceph-mon[34537]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 819 MiB/s wr, 1.84k op/s 2024-05-11T12:30:07.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.838+0000 7f4f89ffb700 1 -- 172.21.15.47:0/1988067712 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f4f980aa210 con 0x55b8f9e43c70 2024-05-11T12:30:07.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.838+0000 7f4f89ffb700 10 client.15594 handle_command_reply: tid=0 2024-05-11T12:30:07.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.838+0000 7f4fb30cab80 1 client.15594 shutdown 2024-05-11T12:30:07.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.838+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 msgr2=0x55b8f9dcd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.838+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9dcd4e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4f98006010 tx=0x7f4f980aa180 comp rx=0 tx=0).stop 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 shutdown_connections 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55b8f9e43c70 0x55b8f9e46060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55b8f9dd35d0 0x55b8f9dcd4e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55b8f9dc3060 0x55b8f9dcfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 --2- 172.21.15.47:0/1988067712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55b8f9dc3980 0x55b8f9dccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 >> 172.21.15.47:0/1988067712 conn(0x55b8f9ce9bf0 msgr2=0x55b8f9dc7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4f89ffb700 0 client.15594 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 shutdown_connections 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.839+0000 7f4fb30cab80 1 -- 172.21.15.47:0/1988067712 wait complete. 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.840+0000 7f4fb30cab80 20 client.15594 trim_cache size 0 max 16384 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:30:07.841 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "64e9ab58-e74f-459b-9064-1824f1944b7e", 2024-05-11T12:30:07.842 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:30:07.842 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:30:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 -- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4f8c0b2960 msgr2=0x7f4f8c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4f8c0b2960 0x7f4f8c0b4e20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00abd0 tx=0x7f4f9c009230 comp rx=0 tx=0).stop 2024-05-11T12:30:07.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 -- 172.21.15.47:0/1881351659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa411cd30 msgr2=0x7f4fa41a8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa411cd30 0x7f4fa41a8800 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f98003ae0 tx=0x7f4f98003bc0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 -- 172.21.15.47:0/1881351659 shutdown_connections 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4fa411cd30 0x7f4fa41a8800 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4fa410aa90 0x7f4fa41ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4f8c0b2960 0x7f4f8c0b4e20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 --2- 172.21.15.47:0/1881351659 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4fa4111d30 0x7f4fa41aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.841+0000 7f4f917fa700 1 -- 172.21.15.47:0/1881351659 >> 172.21.15.47:0/1881351659 conn(0x7f4fa406e6c0 msgr2=0x7f4fa411ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.842+0000 7f4f917fa700 1 -- 172.21.15.47:0/1881351659 shutdown_connections 2024-05-11T12:30:07.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:07.843+0000 7f4f917fa700 1 -- 172.21.15.47:0/1881351659 wait complete. 2024-05-11T12:30:09.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:09 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:30:09.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:09 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:09.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:09 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:30:09.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:09 smithi050 ceph-mon[34072]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 2.04k op/s 2024-05-11T12:30:09.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:09 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:30:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:09 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:09 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:30:09.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:09 smithi047 ceph-mon[30375]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 2.04k op/s 2024-05-11T12:30:09.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:09 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:30:09.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:09 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:09.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:09 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:30:09.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:09 smithi064 ceph-mon[34537]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 2.04k op/s 2024-05-11T12:30:10.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:30:10.230 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:10 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:10 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:10.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:10 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:11.036 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:30:11.485 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:11 smithi047 ceph-mon[30375]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 743 MiB/s wr, 1.88k op/s 2024-05-11T12:30:11.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:11 smithi064 ceph-mon[34537]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 743 MiB/s wr, 1.88k op/s 2024-05-11T12:30:11.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:11 smithi050 ceph-mon[34072]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 743 MiB/s wr, 1.88k op/s 2024-05-11T12:30:13.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:13 smithi047 ceph-mon[30375]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 902 MiB/s wr, 2.05k op/s 2024-05-11T12:30:13.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:13 smithi064 ceph-mon[34537]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 902 MiB/s wr, 2.05k op/s 2024-05-11T12:30:13.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:13 smithi050 ceph-mon[34072]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 902 MiB/s wr, 2.05k op/s 2024-05-11T12:30:14.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:14 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:30:14.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:14 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:30:14.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:14 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:30:15.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:15 smithi047 ceph-mon[30375]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 725 MiB/s wr, 1.60k op/s 2024-05-11T12:30:15.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:15 smithi064 ceph-mon[34537]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 725 MiB/s wr, 1.60k op/s 2024-05-11T12:30:15.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:15 smithi050 ceph-mon[34072]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 725 MiB/s wr, 1.60k op/s 2024-05-11T12:30:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.118+0000 7f9ca4707700 1 -- 172.21.15.47:0/3987942863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c9c11cd30 msgr2=0x7f9c9c11f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.118+0000 7f9ca4707700 1 --2- 172.21.15.47:0/3987942863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c9c11cd30 0x7f9c9c11f120 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400b0a0 tx=0x7f9c9400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.149+0000 7f9ca4707700 1 -- 172.21.15.47:0/3987942863 shutdown_connections 2024-05-11T12:30:17.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.149+0000 7f9ca4707700 1 --2- 172.21.15.47:0/3987942863 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c9c11cd30 0x7f9c9c11f120 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.149+0000 7f9ca4707700 1 --2- 172.21.15.47:0/3987942863 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c9c111d30 0x7f9c9c10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.149+0000 7f9ca4707700 1 --2- 172.21.15.47:0/3987942863 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c10aa90 0x7f9c9c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.150+0000 7f9ca4707700 1 -- 172.21.15.47:0/3987942863 >> 172.21.15.47:0/3987942863 conn(0x7f9c9c06e6c0 msgr2=0x7f9c9c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:17.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.159+0000 7f9ca4707700 1 -- 172.21.15.47:0/3987942863 shutdown_connections 2024-05-11T12:30:17.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.159+0000 7f9ca4707700 1 -- 172.21.15.47:0/3987942863 wait complete. 2024-05-11T12:30:17.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.160+0000 7f9ca4707700 1 Processor -- start 2024-05-11T12:30:17.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.160+0000 7f9ca4707700 1 -- start start 2024-05-11T12:30:17.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.161+0000 7f9ca4707700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c10aa90 0x7f9c9c11b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.161+0000 7f9ca4707700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c9c111d30 0x7f9c9c11b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.161+0000 7f9ca4707700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c9c11cd30 0x7f9c9c1144a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.161+0000 7f9ca4707700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1226b0 con 0x7f9c9c111d30 2024-05-11T12:30:17.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.161+0000 7f9ca4707700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c122530 con 0x7f9c9c11cd30 2024-05-11T12:30:17.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.161+0000 7f9ca4707700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c122830 con 0x7f9c9c10aa90 2024-05-11T12:30:17.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.163+0000 7f9ca2ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c9c11cd30 0x7f9c9c1144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.164+0000 7f9ca2ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c9c11cd30 0x7f9c9c1144a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:44704/0 (socket says 172.21.15.47:44704) 2024-05-11T12:30:17.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.164+0000 7f9ca2ca4700 1 -- 172.21.15.47:0/467393859 learned_addr learned my addr 172.21.15.47:0/467393859 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.164+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c10aa90 0x7f9c9c11b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.166+0000 7f9ca24a3700 1 -- 172.21.15.47:0/467393859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c9c11cd30 msgr2=0x7f9c9c1144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.166+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c9c11cd30 0x7f9c9c1144a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.166+0000 7f9ca24a3700 1 -- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c9c111d30 msgr2=0x7f9c9c11b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.166+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c9c111d30 0x7f9c9c11b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.166+0000 7f9ca24a3700 1 -- 172.21.15.47:0/467393859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c114da0 con 0x7f9c9c10aa90 2024-05-11T12:30:17.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.170+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c10aa90 0x7f9c9c11b260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006fd0 tx=0x7f9c9800b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.172+0000 7f9c937fe700 1 -- 172.21.15.47:0/467393859 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98015020 con 0x7f9c9c10aa90 2024-05-11T12:30:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.172+0000 7f9c937fe700 1 -- 172.21.15.47:0/467393859 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c98011ce0 con 0x7f9c9c10aa90 2024-05-11T12:30:17.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.172+0000 7f9c937fe700 1 -- 172.21.15.47:0/467393859 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98010ce0 con 0x7f9c9c10aa90 2024-05-11T12:30:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.173+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c115090 con 0x7f9c9c10aa90 2024-05-11T12:30:17.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.173+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c119020 con 0x7f9c9c10aa90 2024-05-11T12:30:17.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.181+0000 7f9caef9db80 1 Processor -- start 2024-05-11T12:30:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.185+0000 7f9caef9db80 1 -- start start 2024-05-11T12:30:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.186+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdca4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.186+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdc9f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.186+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9fa60 0x5559bdcab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.186+0000 7f9caef9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5559bdbf1860 con 0x5559bdc9f0a0 2024-05-11T12:30:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.186+0000 7f9caef9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5559bdb907c0 con 0x5559bdca4ad0 2024-05-11T12:30:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.186+0000 7f9caef9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5559bdbed820 con 0x5559bdc9fa60 2024-05-11T12:30:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.187+0000 7f9ca1ca2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9fa60 0x5559bdcab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.187+0000 7f9ca1ca2700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9fa60 0x5559bdcab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39510/0 (socket says 172.21.15.47:39510) 2024-05-11T12:30:17.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.187+0000 7f9ca1ca2700 1 -- 172.21.15.47:0/2713451849 learned_addr learned my addr 172.21.15.47:0/2713451849 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:17.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.190+0000 7f9ca2ca4700 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdca4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.190+0000 7f9ca1ca2700 1 -- 172.21.15.47:0/2713451849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 msgr2=0x5559bdca4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:17.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.191+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdca4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.191+0000 7f9ca1ca2700 1 -- 172.21.15.47:0/2713451849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 msgr2=0x5559bdc9f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:17.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.191+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdc9f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.191+0000 7f9ca1ca2700 1 -- 172.21.15.47:0/2713451849 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bdb3e8d0 con 0x5559bdc9fa60 2024-05-11T12:30:17.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.191+0000 7f9ca2ca4700 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdca4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:30:17.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.195+0000 7f9c937fe700 1 -- 172.21.15.47:0/467393859 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9c98018020 con 0x7f9c9c10aa90 2024-05-11T12:30:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.197+0000 7f9c937fe700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9c880b2900 0x7f9c880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.197+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9fa60 0x5559bdcab890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00aa00 tx=0x7f9c8c00ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51b383f07a9ea4b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:17.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.198+0000 7f9c937fe700 1 -- 172.21.15.47:0/467393859 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9c980a7560 con 0x7f9c9c10aa90 2024-05-11T12:30:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.199+0000 7f9ca2ca4700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9c880b2900 0x7f9c880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.200+0000 7f9c90ff9700 1 -- 172.21.15.47:0/2713451849 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00cbb0 con 0x5559bdc9fa60 2024-05-11T12:30:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.200+0000 7f9c90ff9700 1 -- 172.21.15.47:0/2713451849 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8c0044d0 con 0x5559bdc9fa60 2024-05-11T12:30:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.200+0000 7f9c90ff9700 1 -- 172.21.15.47:0/2713451849 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c00f450 con 0x5559bdc9fa60 2024-05-11T12:30:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.201+0000 7f9caef9db80 1 -- 172.21.15.47:0/2713451849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9fa60 msgr2=0x5559bdcab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.201+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9fa60 0x5559bdcab890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00aa00 tx=0x7f9c8c00ad10 comp rx=0 tx=0).stop 2024-05-11T12:30:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.209+0000 7f9caef9db80 1 -- 172.21.15.47:0/2713451849 shutdown_connections 2024-05-11T12:30:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.209+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9fa60 0x5559bdcab890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.209+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdca4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.209+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2713451849 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdc9f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.209+0000 7f9caef9db80 1 -- 172.21.15.47:0/2713451849 >> 172.21.15.47:0/2713451849 conn(0x5559bdbcbe60 msgr2=0x5559bdc99990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:17.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.211+0000 7f9caef9db80 1 -- 172.21.15.47:0/2713451849 shutdown_connections 2024-05-11T12:30:17.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.212+0000 7f9ca2ca4700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9c880b2900 0x7f9c880b4dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009e60 tx=0x7f9c94005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:17.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.213+0000 7f9caef9db80 1 -- 172.21.15.47:0/2713451849 wait complete. 2024-05-11T12:30:17.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.214+0000 7f9caef9db80 1 -- start start 2024-05-11T12:30:17.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.214+0000 7f9caef9db80 1 Processor -- start 2024-05-11T12:30:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.224+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdcab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.224+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdca88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.224+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdca4ad0 0x5559bdca8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.224+0000 7f9caef9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5559bdcabe90 con 0x5559bdca4ad0 2024-05-11T12:30:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.224+0000 7f9caef9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5559bdb57c30 con 0x5559bdc9fa60 2024-05-11T12:30:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.224+0000 7f9caef9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5559bdcac010 con 0x5559bdc9f0a0 2024-05-11T12:30:17.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.224+0000 7f9caef9db80 10 client.0 _ll_register_callbacks cb 0x5559bdb89920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.229+0000 7f9ca2ca4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdca4ad0 0x5559bdca8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.229+0000 7f9ca2ca4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdca4ad0 0x5559bdca8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45424/0 (socket says 172.21.15.47:45424) 2024-05-11T12:30:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.229+0000 7f9ca2ca4700 1 -- 172.21.15.47:0/671507635 learned_addr learned my addr 172.21.15.47:0/671507635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.229+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdca88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.229+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdcab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.233+0000 7f9ca1ca2700 1 -- 172.21.15.47:0/671507635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 msgr2=0x5559bdcab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.233+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdcab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.233+0000 7f9ca1ca2700 1 -- 172.21.15.47:0/671507635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdca4ad0 msgr2=0x5559bdca8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.233+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdca4ad0 0x5559bdca8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.233+0000 7f9ca1ca2700 1 -- 172.21.15.47:0/671507635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bdb8d690 con 0x5559bdc9fa60 2024-05-11T12:30:17.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.239+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdca88f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c000c00 tx=0x7f9c8c004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37bd4971f976d88 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:17.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.240+0000 7f9c85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:30:17.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.240+0000 7f9c85ffb700 1 -- 172.21.15.47:0/671507635 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c028060 con 0x5559bdc9fa60 2024-05-11T12:30:17.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.243+0000 7f9c85ffb700 1 -- 172.21.15.47:0/671507635 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9c8c003d10 con 0x5559bdc9fa60 2024-05-11T12:30:17.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.245+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c8c0044d0 con 0x5559bdc9fa60 2024-05-11T12:30:17.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.247+0000 7f9c85ffb700 1 -- 172.21.15.47:0/671507635 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9c8c00c9b0 con 0x5559bdc9fa60 2024-05-11T12:30:17.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.247+0000 7f9caef9db80 10 client.25181 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:17.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.247+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5559bdcb66b0 con 0x5559bdc9fa60 2024-05-11T12:30:17.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.249+0000 7f9c85ffb700 1 -- 172.21.15.47:0/671507635 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c8c0044d0 con 0x5559bdc9fa60 2024-05-11T12:30:17.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.249+0000 7f9c85ffb700 1 -- 172.21.15.47:0/671507635 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f9c8c00fca0 con 0x5559bdc9fa60 2024-05-11T12:30:17.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.249+0000 7f9caef9db80 10 client.25181 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:17.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.249+0000 7f9caef9db80 10 client.25181 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:17.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.250+0000 7f9caef9db80 20 client.25181 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:17.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.250+0000 7f9caef9db80 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdcbaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.250+0000 7f9caef9db80 4 client.25181 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:30:17.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.250+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559bdbff240 con 0x5559bdcb8b80 2024-05-11T12:30:17.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.255+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdcbaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.262+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdcbaf70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95dccd5b4efc82 server_cookie=2f1da7844b4bed6e in_seq=0 out_seq=0 2024-05-11T12:30:17.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.262+0000 7f9c85ffb700 10 client.25181 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:17.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.266+0000 7f9c85ffb700 1 -- 172.21.15.47:0/671507635 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559bdbff240 con 0x5559bdcb8b80 2024-05-11T12:30:17.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.266+0000 7f9c85ffb700 10 client.25181 handle_command_reply: tid=0 2024-05-11T12:30:17.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.266+0000 7f9caef9db80 1 client.25181 shutdown 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.267+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 msgr2=0x5559bdca88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.267+0000 7f9caef9db80 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdca88f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c000c00 tx=0x7f9c8c004390 comp rx=0 tx=0).stop 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.268+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 shutdown_connections 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.268+0000 7f9caef9db80 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdcbaf70 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.268+0000 7f9caef9db80 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdca4ad0 0x5559bdca8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.268+0000 7f9caef9db80 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdcab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.268+0000 7f9c85ffb700 0 client.25181 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.268+0000 7f9caef9db80 1 --2- 172.21.15.47:0/671507635 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdca88f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.268+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 >> 172.21.15.47:0/671507635 conn(0x5559bdbcbe60 msgr2=0x5559bdc94460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:17.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.269+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 shutdown_connections 2024-05-11T12:30:17.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.270+0000 7f9caef9db80 1 -- 172.21.15.47:0/671507635 wait complete. 2024-05-11T12:30:17.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.270+0000 7f9caef9db80 20 client.25181 trim_cache size 0 max 16384 2024-05-11T12:30:17.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.333+0000 7f9caef9db80 1 Processor -- start 2024-05-11T12:30:17.335 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:17 smithi047 ceph-mon[30375]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 768 MiB/s wr, 1.69k op/s 2024-05-11T12:30:17.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.341+0000 7f9caef9db80 1 -- start start 2024-05-11T12:30:17.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.342+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdca4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.342+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9fa60 0x5559bdc93bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.342+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdc94110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.343+0000 7f9caef9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1c36d0 con 0x5559bdc9fa60 2024-05-11T12:30:17.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.343+0000 7f9caef9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1c5f50 con 0x5559bdca4ad0 2024-05-11T12:30:17.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.343+0000 7f9caef9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1c3d40 con 0x5559bdc9f0a0 2024-05-11T12:30:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.347+0000 7f9ca2ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdc94110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.347+0000 7f9ca2ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdc94110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:44724/0 (socket says 172.21.15.47:44724) 2024-05-11T12:30:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.347+0000 7f9ca2ca4700 1 -- 172.21.15.47:0/3412225304 learned_addr learned my addr 172.21.15.47:0/3412225304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.350+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdca4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.347+0000 7f9ca1ca2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9fa60 0x5559bdc93bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.352+0000 7f9ca2ca4700 1 -- 172.21.15.47:0/3412225304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 msgr2=0x5559bdca4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.352+0000 7f9ca2ca4700 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdca4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.352+0000 7f9ca2ca4700 1 -- 172.21.15.47:0/3412225304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9fa60 msgr2=0x5559bdc93bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.352+0000 7f9ca2ca4700 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9fa60 0x5559bdc93bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.352+0000 7f9ca2ca4700 1 -- 172.21.15.47:0/3412225304 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559bdc9f6d0 con 0x5559bdca4ad0 2024-05-11T12:30:17.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.354+0000 7f9ca2ca4700 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdc94110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c94003f10 tx=0x7f9c9401f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929a499f544ad91 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:17.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.360+0000 7f9c87fff700 1 -- 172.21.15.47:0/3412225304 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9400bd70 con 0x5559bdca4ad0 2024-05-11T12:30:17.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.360+0000 7f9c87fff700 1 -- 172.21.15.47:0/3412225304 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9402f030 con 0x5559bdca4ad0 2024-05-11T12:30:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.360+0000 7f9c87fff700 1 -- 172.21.15.47:0/3412225304 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94007a30 con 0x5559bdca4ad0 2024-05-11T12:30:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.361+0000 7f9caef9db80 1 -- 172.21.15.47:0/3412225304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 msgr2=0x5559bdc94110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.361+0000 7f9caef9db80 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdc94110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9c94003f10 tx=0x7f9c9401f070 comp rx=0 tx=0).stop 2024-05-11T12:30:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.364+0000 7f9caef9db80 1 -- 172.21.15.47:0/3412225304 shutdown_connections 2024-05-11T12:30:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.364+0000 7f9caef9db80 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdca4ad0 0x5559bdc94110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.364+0000 7f9caef9db80 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdc9f0a0 0x5559bdca4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.364+0000 7f9caef9db80 1 --2- 172.21.15.47:0/3412225304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9fa60 0x5559bdc93bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.364+0000 7f9caef9db80 1 -- 172.21.15.47:0/3412225304 >> 172.21.15.47:0/3412225304 conn(0x5559bdbcbe60 msgr2=0x5559bdc9e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:17.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.366+0000 7f9caef9db80 1 -- 172.21.15.47:0/3412225304 shutdown_connections 2024-05-11T12:30:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.366+0000 7f9caef9db80 1 -- 172.21.15.47:0/3412225304 wait complete. 2024-05-11T12:30:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.366+0000 7f9caef9db80 1 -- start start 2024-05-11T12:30:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.366+0000 7f9caef9db80 1 Processor -- start 2024-05-11T12:30:17.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.367+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdcbc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.367+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdcbc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.367+0000 7f9caef9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdca4ad0 0x5559bdd1c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.367+0000 7f9caef9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1c4410 con 0x5559bdc9f0a0 2024-05-11T12:30:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.367+0000 7f9caef9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1c5880 con 0x5559bdc9fa60 2024-05-11T12:30:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.367+0000 7f9caef9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c1c4ae0 con 0x5559bdca4ad0 2024-05-11T12:30:17.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.367+0000 7f9caef9db80 10 client.0 _ll_register_callbacks cb 0x5559bdb89920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.369+0000 7f9ca24a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdcbc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.369+0000 7f9ca24a3700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdcbc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45440/0 (socket says 172.21.15.47:45440) 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.369+0000 7f9ca24a3700 1 -- 172.21.15.47:0/2070494720 learned_addr learned my addr 172.21.15.47:0/2070494720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.369+0000 7f9ca24a3700 1 -- 172.21.15.47:0/2070494720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdca4ad0 msgr2=0x5559bdd1c070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.369+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdca4ad0 0x5559bdd1c070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.369+0000 7f9ca24a3700 1 -- 172.21.15.47:0/2070494720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 msgr2=0x5559bdcbc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.369+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdcbc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.370+0000 7f9ca24a3700 1 -- 172.21.15.47:0/2070494720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559bdd1c660 con 0x5559bdc9f0a0 2024-05-11T12:30:17.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.370+0000 7f9ca24a3700 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdcbc370 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009fd0 tx=0x7f9c980b38d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0d4fde17b86f04 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.371+0000 7f9c85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.371+0000 7f9c85ffb700 1 -- 172.21.15.47:0/2070494720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c980a7760 con 0x5559bdc9f0a0 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.372+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c9402f030 con 0x5559bdc9f0a0 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.372+0000 7f9c85ffb700 1 -- 172.21.15.47:0/2070494720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9c980b3940 con 0x5559bdc9f0a0 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.373+0000 7f9c85ffb700 1 -- 172.21.15.47:0/2070494720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9c980a7900 con 0x5559bdc9f0a0 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.373+0000 7f9caef9db80 10 client.15606 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.373+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5559bdd23a30 con 0x5559bdc9f0a0 2024-05-11T12:30:17.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.374+0000 7f9c85ffb700 1 -- 172.21.15.47:0/2070494720 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c9402f030 con 0x5559bdc9f0a0 2024-05-11T12:30:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.375+0000 7f9c85ffb700 1 -- 172.21.15.47:0/2070494720 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f9c98022060 con 0x5559bdc9f0a0 2024-05-11T12:30:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.375+0000 7f9caef9db80 10 client.15606 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.375+0000 7f9caef9db80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.375+0000 7f9caef9db80 20 client.15606 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.375+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdd26120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.375+0000 7f9caef9db80 4 client.15606 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:30:17.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.375+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559bdbf92e0 con 0x5559bdcb8b80 2024-05-11T12:30:17.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.380+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdd26120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:17.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.403+0000 7f9ca1ca2700 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdd26120 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef91fe8af5b4c07 server_cookie=bf770e2ac4aceb04 in_seq=0 out_seq=0 2024-05-11T12:30:17.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.403+0000 7f9c85ffb700 10 client.15606 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:17.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.408+0000 7f9c85ffb700 1 -- 172.21.15.47:0/2070494720 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5559bdbf92e0 con 0x5559bdcb8b80 2024-05-11T12:30:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.408+0000 7f9c85ffb700 10 client.15606 handle_command_reply: tid=0 2024-05-11T12:30:17.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.409+0000 7f9caef9db80 1 client.15606 shutdown 2024-05-11T12:30:17.420 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:30:17.420 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 msgr2=0x5559bdcbc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdcbc370 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009fd0 tx=0x7f9c980b38d0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 shutdown_connections 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5559bdcb8b80 0x5559bdd26120 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5559bdca4ad0 0x5559bdd1c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5559bdc9f0a0 0x5559bdcbc370 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 --2- 172.21.15.47:0/2070494720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5559bdc9fa60 0x5559bdcbc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9c85ffb700 0 client.15606 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 >> 172.21.15.47:0/2070494720 conn(0x5559bdbcbe60 msgr2=0x5559bdc9e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.416+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 shutdown_connections 2024-05-11T12:30:17.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.417+0000 7f9caef9db80 1 -- 172.21.15.47:0/2070494720 wait complete. 2024-05-11T12:30:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.417+0000 7f9caef9db80 20 client.15606 trim_cache size 0 max 16384 2024-05-11T12:30:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9c880b2900 msgr2=0x7f9c880b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9c880b2900 0x7f9c880b4dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009e60 tx=0x7f9c94005040 comp rx=0 tx=0).stop 2024-05-11T12:30:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c10aa90 msgr2=0x7f9c9c11b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:17.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c10aa90 0x7f9c9c11b260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c98006fd0 tx=0x7f9c9800b730 comp rx=0 tx=0).stop 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 shutdown_connections 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9c880b2900 0x7f9c880b4dc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c9c11cd30 0x7f9c9c1144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9c9c10aa90 0x7f9c9c11b260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 --2- 172.21.15.47:0/467393859 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9c9c111d30 0x7f9c9c11b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 >> 172.21.15.47:0/467393859 conn(0x7f9c9c06e6c0 msgr2=0x7f9c9c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.418+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 shutdown_connections 2024-05-11T12:30:17.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:17.419+0000 7f9ca4707700 1 -- 172.21.15.47:0/467393859 wait complete. 2024-05-11T12:30:17.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:17 smithi064 ceph-mon[34537]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 768 MiB/s wr, 1.69k op/s 2024-05-11T12:30:17.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:17 smithi050 ceph-mon[34072]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 768 MiB/s wr, 1.69k op/s 2024-05-11T12:30:19.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:19 smithi047 ceph-mon[30375]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 836 MiB/s wr, 1.65k op/s 2024-05-11T12:30:19.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:19 smithi064 ceph-mon[34537]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 836 MiB/s wr, 1.65k op/s 2024-05-11T12:30:19.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:19 smithi050 ceph-mon[34072]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 836 MiB/s wr, 1.65k op/s 2024-05-11T12:30:19.944 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:30:19.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:30:20.962 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:30:21.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:21 smithi047 ceph-mon[30375]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 771 MiB/s wr, 1.46k op/s 2024-05-11T12:30:21.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:21 smithi064 ceph-mon[34537]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 771 MiB/s wr, 1.46k op/s 2024-05-11T12:30:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:21 smithi050 ceph-mon[34072]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 771 MiB/s wr, 1.46k op/s 2024-05-11T12:30:23.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:23 smithi064 ceph-mon[34537]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 1.90k op/s 2024-05-11T12:30:23.701 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:23 smithi050 ceph-mon[34072]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 1.90k op/s 2024-05-11T12:30:23.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:23 smithi047 ceph-mon[30375]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 902 MiB/s wr, 1.90k op/s 2024-05-11T12:30:25.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:25 smithi047 ceph-mon[30375]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 754 MiB/s wr, 1.80k op/s 2024-05-11T12:30:25.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:25 smithi064 ceph-mon[34537]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 754 MiB/s wr, 1.80k op/s 2024-05-11T12:30:25.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:25 smithi050 ceph-mon[34072]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 754 MiB/s wr, 1.80k op/s 2024-05-11T12:30:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.010+0000 7fe53d640700 1 -- 172.21.15.47:0/882661809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe53811cd30 msgr2=0x7fe53811f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.010+0000 7fe53d640700 1 --2- 172.21.15.47:0/882661809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe53811cd30 0x7fe53811f120 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe53000b0a0 tx=0x7fe53000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.012+0000 7fe53d640700 1 -- 172.21.15.47:0/882661809 shutdown_connections 2024-05-11T12:30:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.012+0000 7fe53d640700 1 --2- 172.21.15.47:0/882661809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe53811cd30 0x7fe53811f120 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.012+0000 7fe53d640700 1 --2- 172.21.15.47:0/882661809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe538111c40 0x7fe53810a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.012+0000 7fe53d640700 1 --2- 172.21.15.47:0/882661809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe5381120b0 0x7fe53810aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.012+0000 7fe53d640700 1 -- 172.21.15.47:0/882661809 >> 172.21.15.47:0/882661809 conn(0x7fe53806e560 msgr2=0x7fe53806e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.021+0000 7fe53d640700 1 -- 172.21.15.47:0/882661809 shutdown_connections 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.021+0000 7fe53d640700 1 -- 172.21.15.47:0/882661809 wait complete. 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.022+0000 7fe53d640700 1 Processor -- start 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.023+0000 7fe53d640700 1 -- start start 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.023+0000 7fe53d640700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538111c40 0x7fe53811b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.023+0000 7fe53d640700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5381120b0 0x7fe53811b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.023+0000 7fe53d640700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53811cd30 0x7fe538114530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.023+0000 7fe53d640700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe538122600 con 0x7fe53811cd30 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.023+0000 7fe53d640700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe538122480 con 0x7fe5381120b0 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.024+0000 7fe53d640700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe538122780 con 0x7fe538111c40 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.026+0000 7fe536ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538111c40 0x7fe53811b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.026+0000 7fe536ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538111c40 0x7fe53811b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39562/0 (socket says 172.21.15.47:39562) 2024-05-11T12:30:26.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.026+0000 7fe536ffd700 1 -- 172.21.15.47:0/96774093 learned_addr learned my addr 172.21.15.47:0/96774093 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.027+0000 7fe536ffd700 1 -- 172.21.15.47:0/96774093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5381120b0 msgr2=0x7fe53811b790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.027+0000 7fe536ffd700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5381120b0 0x7fe53811b790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.027+0000 7fe536ffd700 1 -- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53811cd30 msgr2=0x7fe538114530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.027+0000 7fe536ffd700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53811cd30 0x7fe538114530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.027+0000 7fe536ffd700 1 -- 172.21.15.47:0/96774093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe538114e30 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.028+0000 7fe536ffd700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538111c40 0x7fe53811b250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b530 tx=0x7fe52800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.029+0000 7fe51ffff700 1 -- 172.21.15.47:0/96774093 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528015020 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.029+0000 7fe51ffff700 1 -- 172.21.15.47:0/96774093 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe528011ce0 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.029+0000 7fe51ffff700 1 -- 172.21.15.47:0/96774093 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528010d40 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.032+0000 7fe53d640700 1 -- 172.21.15.47:0/96774093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5381150c0 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.032+0000 7fe53d640700 1 -- 172.21.15.47:0/96774093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe538119010 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.033+0000 7fe53d640700 1 -- 172.21.15.47:0/96774093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53806f9c0 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.043+0000 7fe51ffff700 1 -- 172.21.15.47:0/96774093 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe528018020 con 0x7fe538111c40 2024-05-11T12:30:26.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.045+0000 7fe51ffff700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe5200b2930 0x7fe5200b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.055+0000 7fe51ffff700 1 -- 172.21.15.47:0/96774093 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fe5280a5c80 con 0x7fe538111c40 2024-05-11T12:30:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.056+0000 7fe51ffff700 1 -- 172.21.15.47:0/96774093 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe5280a9d90 con 0x7fe538111c40 2024-05-11T12:30:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.059+0000 7fe5367fc700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe5200b2930 0x7fe5200b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:26.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.073+0000 7fe5367fc700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe5200b2930 0x7fe5200b4df0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe538115aa0 tx=0x7fe52c006cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:26.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.642+0000 7fe53d640700 1 -- 172.21.15.47:0/96774093 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe538119300 con 0x7fe538111c40 2024-05-11T12:30:26.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.646+0000 7fe51ffff700 1 -- 172.21.15.47:0/96774093 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fe5280ada40 con 0x7fe538111c40 2024-05-11T12:30:26.647 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:30:26.647 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:30:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.657+0000 7fe51dffb700 1 -- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe5200b2930 msgr2=0x7fe5200b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.657+0000 7fe51dffb700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe5200b2930 0x7fe5200b4df0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe538115aa0 tx=0x7fe52c006cb0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.657+0000 7fe51dffb700 1 -- 172.21.15.47:0/96774093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538111c40 msgr2=0x7fe53811b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.657+0000 7fe51dffb700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538111c40 0x7fe53811b250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe52800b530 tx=0x7fe52800b8f0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.658+0000 7fe51dffb700 1 -- 172.21.15.47:0/96774093 shutdown_connections 2024-05-11T12:30:26.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.658+0000 7fe51dffb700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe53811cd30 0x7fe538114530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.658+0000 7fe51dffb700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe538111c40 0x7fe53811b250 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.658+0000 7fe51dffb700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe5200b2930 0x7fe5200b4df0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.658+0000 7fe51dffb700 1 --2- 172.21.15.47:0/96774093 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe5381120b0 0x7fe53811b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:26.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.658+0000 7fe51dffb700 1 -- 172.21.15.47:0/96774093 >> 172.21.15.47:0/96774093 conn(0x7fe53806e560 msgr2=0x7fe538071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:26.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.663+0000 7fe51dffb700 1 -- 172.21.15.47:0/96774093 shutdown_connections 2024-05-11T12:30:26.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:26.666+0000 7fe51dffb700 1 -- 172.21.15.47:0/96774093 wait complete. 2024-05-11T12:30:26.671 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:30:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:27 smithi047 ceph-mon[30375]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 760 MiB/s wr, 1.75k op/s 2024-05-11T12:30:27.314 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:27 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/96774093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:30:27.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:27 smithi064 ceph-mon[34537]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 760 MiB/s wr, 1.75k op/s 2024-05-11T12:30:27.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:27 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/96774093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:30:27.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:27 smithi050 ceph-mon[34072]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 760 MiB/s wr, 1.75k op/s 2024-05-11T12:30:27.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:27 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/96774093' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:30:28.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:30:29.286 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:30:29.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:29 smithi064 ceph-mon[34537]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 902 MiB/s wr, 1.89k op/s 2024-05-11T12:30:29.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:29 smithi050 ceph-mon[34072]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 902 MiB/s wr, 1.89k op/s 2024-05-11T12:30:29.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:29 smithi047 ceph-mon[30375]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 902 MiB/s wr, 1.89k op/s 2024-05-11T12:30:31.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:31 smithi064 ceph-mon[34537]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 MiB/s wr, 1.67k op/s 2024-05-11T12:30:31.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:31 smithi047 ceph-mon[30375]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 MiB/s wr, 1.67k op/s 2024-05-11T12:30:31.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:31 smithi050 ceph-mon[34072]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 MiB/s wr, 1.67k op/s 2024-05-11T12:30:33.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:33 smithi064 ceph-mon[34537]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 1.81k op/s 2024-05-11T12:30:33.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:33 smithi050 ceph-mon[34072]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 1.81k op/s 2024-05-11T12:30:33.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:33 smithi047 ceph-mon[30375]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 1.81k op/s 2024-05-11T12:30:35.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.235+0000 7f9e7b35d700 1 -- 172.21.15.47:0/521944615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e7410aa90 msgr2=0x7f9e7410af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.235+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/521944615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e7410aa90 0x7f9e7410af10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9e640098f0 tx=0x7f9e64009c00 comp rx=0 tx=0).stop 2024-05-11T12:30:35.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.236+0000 7f9e7b35d700 1 -- 172.21.15.47:0/521944615 shutdown_connections 2024-05-11T12:30:35.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.236+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/521944615 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7411cd30 0x7f9e7411f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.236+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/521944615 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e74111d30 0x7f9e7410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.236+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/521944615 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e7410aa90 0x7f9e7410af10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.236+0000 7f9e7b35d700 1 -- 172.21.15.47:0/521944615 >> 172.21.15.47:0/521944615 conn(0x7f9e7406e6c0 msgr2=0x7f9e7406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:35.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.237+0000 7f9e7b35d700 1 -- 172.21.15.47:0/521944615 shutdown_connections 2024-05-11T12:30:35.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.238+0000 7f9e7b35d700 1 -- 172.21.15.47:0/521944615 wait complete. 2024-05-11T12:30:35.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.239+0000 7f9e7b35d700 1 Processor -- start 2024-05-11T12:30:35.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.239+0000 7f9e7b35d700 1 -- start start 2024-05-11T12:30:35.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.240+0000 7f9e7b35d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e7410aa90 0x7f9e741ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.240+0000 7f9e7b35d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e74111d30 0x7f9e741aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.240+0000 7f9e7b35d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7411cd30 0x7f9e741a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.240+0000 7f9e7b35d700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74122640 con 0x7f9e7410aa90 2024-05-11T12:30:35.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.240+0000 7f9e7b35d700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741224c0 con 0x7f9e7411cd30 2024-05-11T12:30:35.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.240+0000 7f9e7b35d700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e741227c0 con 0x7f9e74111d30 2024-05-11T12:30:35.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.243+0000 7f9e798fa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7411cd30 0x7f9e741a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.243+0000 7f9e798fa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7411cd30 0x7f9e741a89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:41990/0 (socket says 172.21.15.47:41990) 2024-05-11T12:30:35.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.243+0000 7f9e798fa700 1 -- 172.21.15.47:0/396403318 learned_addr learned my addr 172.21.15.47:0/396403318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:35.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.243+0000 7f9e790f9700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e7410aa90 0x7f9e741ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.243+0000 7f9e788f8700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e74111d30 0x7f9e741aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.244+0000 7f9e790f9700 1 -- 172.21.15.47:0/396403318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e74111d30 msgr2=0x7f9e741aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.244+0000 7f9e790f9700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e74111d30 0x7f9e741aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.244+0000 7f9e790f9700 1 -- 172.21.15.47:0/396403318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7411cd30 msgr2=0x7f9e741a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.244+0000 7f9e790f9700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7411cd30 0x7f9e741a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.244+0000 7f9e790f9700 1 -- 172.21.15.47:0/396403318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e741a9100 con 0x7f9e7410aa90 2024-05-11T12:30:35.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.245+0000 7f9e790f9700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e7410aa90 0x7f9e741ae790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000d950 tx=0x7f9e7000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:35.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.247+0000 7f9e6a7fc700 1 -- 172.21.15.47:0/396403318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70017070 con 0x7f9e7410aa90 2024-05-11T12:30:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.247+0000 7f9e6a7fc700 1 -- 172.21.15.47:0/396403318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e70009de0 con 0x7f9e7410aa90 2024-05-11T12:30:35.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.247+0000 7f9e6a7fc700 1 -- 172.21.15.47:0/396403318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70010e20 con 0x7f9e7410aa90 2024-05-11T12:30:35.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.251+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e741a93f0 con 0x7f9e7410aa90 2024-05-11T12:30:35.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.251+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e741b5660 con 0x7f9e7410aa90 2024-05-11T12:30:35.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.259+0000 7f9e6a7fc700 1 -- 172.21.15.47:0/396403318 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9e7001a070 con 0x7f9e7410aa90 2024-05-11T12:30:35.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.261+0000 7f9e6a7fc700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9e600b2a00 0x7f9e600b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.263+0000 7f9e6a7fc700 1 -- 172.21.15.47:0/396403318 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9e700a5620 con 0x7f9e7410aa90 2024-05-11T12:30:35.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.265+0000 7f9e788f8700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9e600b2a00 0x7f9e600b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.267+0000 7f9e788f8700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9e600b2a00 0x7f9e600b4ec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9e7411f4c0 tx=0x7f9e64019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:35.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 Processor -- start 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 -- start start 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c86a0 0x5583f37c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c3b30 0x5583f37c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5583f3715940 con 0x5583f37c7d80 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5583f367ada0 con 0x5583f37c3b30 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.269+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583f3711900 con 0x5583f37c86a0 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.270+0000 7f9e798fa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c3b30 0x5583f37c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.270+0000 7f9e798fa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c3b30 0x5583f37c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:41996/0 (socket says 172.21.15.47:41996) 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.270+0000 7f9e798fa700 1 -- 172.21.15.47:0/764464245 learned_addr learned my addr 172.21.15.47:0/764464245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:35.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.271+0000 7f9e790f9700 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.271+0000 7f9e798fa700 1 -- 172.21.15.47:0/764464245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c86a0 msgr2=0x5583f37c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.271+0000 7f9e798fa700 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c86a0 0x5583f37c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.271+0000 7f9e798fa700 1 -- 172.21.15.47:0/764464245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 msgr2=0x5583f37c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.271+0000 7f9e798fa700 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.271+0000 7f9e798fa700 1 -- 172.21.15.47:0/764464245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f36628d0 con 0x5583f37c3b30 2024-05-11T12:30:35.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.274+0000 7f9e790f9700 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:30:35.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.277+0000 7f9e798fa700 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c3b30 0x5583f37c3f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009490 tx=0x7f9e6c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee0564298fe4e3a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:35.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.280+0000 7f9e5f7fe700 1 -- 172.21.15.47:0/764464245 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c03a020 con 0x5583f37c3b30 2024-05-11T12:30:35.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.281+0000 7f9e5f7fe700 1 -- 172.21.15.47:0/764464245 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e6c0368a0 con 0x5583f37c3b30 2024-05-11T12:30:35.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.281+0000 7f9e5f7fe700 1 -- 172.21.15.47:0/764464245 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c035910 con 0x5583f37c3b30 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.283+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/764464245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c3b30 msgr2=0x5583f37c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.283+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c3b30 0x5583f37c3f90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009490 tx=0x7f9e6c009920 comp rx=0 tx=0).stop 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.283+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/764464245 shutdown_connections 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.283+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c3b30 0x5583f37c3f90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.283+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.283+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/764464245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c86a0 0x5583f37c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.283+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/764464245 >> 172.21.15.47:0/764464245 conn(0x5583f36efb70 msgr2=0x5583f37bdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.284+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/764464245 shutdown_connections 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.284+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/764464245 wait complete. 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.284+0000 7f9e85bf3b80 1 -- start start 2024-05-11T12:30:35.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.284+0000 7f9e85bf3b80 1 Processor -- start 2024-05-11T12:30:35.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.285+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.285+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.285+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.286+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5583f37d0270 con 0x5583f37c7d80 2024-05-11T12:30:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.286+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5583f36ceed0 con 0x5583f37c86a0 2024-05-11T12:30:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.286+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5583f37d03f0 con 0x5583f37c3b30 2024-05-11T12:30:35.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.286+0000 7f9e85bf3b80 10 client.0 _ll_register_callbacks cb 0x5583f36ad950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.287+0000 7f9e790f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.287+0000 7f9e790f9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40944/0 (socket says 172.21.15.47:40944) 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.287+0000 7f9e790f9700 1 -- 172.21.15.47:0/798394078 learned_addr learned my addr 172.21.15.47:0/798394078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.287+0000 7f9e788f8700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.288+0000 7f9e798fa700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.289+0000 7f9e788f8700 1 -- 172.21.15.47:0/798394078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 msgr2=0x5583f37cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.289+0000 7f9e788f8700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.289+0000 7f9e788f8700 1 -- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 msgr2=0x5583f37cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.289+0000 7f9e788f8700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.289+0000 7f9e788f8700 1 -- 172.21.15.47:0/798394078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f36cc760 con 0x5583f37c7d80 2024-05-11T12:30:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.289+0000 7f9e790f9700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:30:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.290+0000 7f9e788f8700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37cc8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009810 tx=0x7f9e64009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd81a725c005e4a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.290+0000 7f9e5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:30:35.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.291+0000 7f9e5cff9700 1 -- 172.21.15.47:0/798394078 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64031060 con 0x5583f37c7d80 2024-05-11T12:30:35.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.292+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6c0368a0 con 0x5583f37c7d80 2024-05-11T12:30:35.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.292+0000 7f9e5cff9700 1 -- 172.21.15.47:0/798394078 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9e640053d0 con 0x5583f37c7d80 2024-05-11T12:30:35.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.303+0000 7f9e5cff9700 1 -- 172.21.15.47:0/798394078 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9e640055d0 con 0x5583f37c7d80 2024-05-11T12:30:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.306+0000 7f9e85bf3b80 10 client.15615 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:35.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.306+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5583f37daba0 con 0x5583f37c7d80 2024-05-11T12:30:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e5cff9700 1 -- 172.21.15.47:0/798394078 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e6c0368a0 con 0x5583f37c7d80 2024-05-11T12:30:35.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e5cff9700 1 -- 172.21.15.47:0/798394078 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f9e64017a00 con 0x5583f37c7d80 2024-05-11T12:30:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e85bf3b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e85bf3b80 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e85bf3b80 20 client.15615 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f37dd5f0 0x5583f37dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e85bf3b80 4 client.15615 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:30:35.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.310+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583f3723320 con 0x5583f37dd5f0 2024-05-11T12:30:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.311+0000 7f9e790f9700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f37dd5f0 0x5583f37dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.313+0000 7f9e790f9700 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f37dd5f0 0x5583f37dfa10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39139fc68a6f80b9 server_cookie=9db7a3390f9e0872 in_seq=0 out_seq=0 2024-05-11T12:30:35.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.313+0000 7f9e5cff9700 10 client.15615 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:35.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.317+0000 7f9e5cff9700 1 -- 172.21.15.47:0/798394078 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583f3723320 con 0x5583f37dd5f0 2024-05-11T12:30:35.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.317+0000 7f9e5cff9700 10 client.15615 handle_command_reply: tid=0 2024-05-11T12:30:35.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.317+0000 7f9e85bf3b80 1 client.15615 shutdown 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.318+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 msgr2=0x5583f37cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.318+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37cc8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009810 tx=0x7f9e64009f50 comp rx=0 tx=0).stop 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.318+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 shutdown_connections 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.318+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f37dd5f0 0x5583f37dfa10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/798394078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c7d80 0x5583f37cc8f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 >> 172.21.15.47:0/798394078 conn(0x5583f36efb70 msgr2=0x5583f37c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e5cff9700 0 client.15615 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:35.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 shutdown_connections 2024-05-11T12:30:35.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/798394078 wait complete. 2024-05-11T12:30:35.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.319+0000 7f9e85bf3b80 20 client.15615 trim_cache size 0 max 16384 2024-05-11T12:30:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.365+0000 7f9e85bf3b80 1 Processor -- start 2024-05-11T12:30:35.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.365+0000 7f9e85bf3b80 1 -- start start 2024-05-11T12:30:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.366+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c3b30 0x5583f37c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.366+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.366+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37cef00 0x5583f37b8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.367+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5800b720 con 0x5583f37c3b30 2024-05-11T12:30:35.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.367+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5800dfa0 con 0x5583f37c86a0 2024-05-11T12:30:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.367+0000 7f9e788f8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.367+0000 7f9e788f8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:42018/0 (socket says 172.21.15.47:42018) 2024-05-11T12:30:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.367+0000 7f9e788f8700 1 -- 172.21.15.47:0/2077308138 learned_addr learned my addr 172.21.15.47:0/2077308138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:35.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.367+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/2077308138 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5800bd90 con 0x5583f37cef00 2024-05-11T12:30:35.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.370+0000 7f9e798fa700 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37cef00 0x5583f37b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.370+0000 7f9e790f9700 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c3b30 0x5583f37c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.370+0000 7f9e788f8700 1 -- 172.21.15.47:0/2077308138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37cef00 msgr2=0x5583f37b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.370+0000 7f9e788f8700 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37cef00 0x5583f37b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.370+0000 7f9e788f8700 1 -- 172.21.15.47:0/2077308138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c3b30 msgr2=0x5583f37c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.370+0000 7f9e788f8700 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c3b30 0x5583f37c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.371+0000 7f9e788f8700 1 -- 172.21.15.47:0/2077308138 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583f37ba980 con 0x5583f37c86a0 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.371+0000 7f9e788f8700 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e64005040 tx=0x7f9e6400b5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a32819eaaf9dc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:35.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.374+0000 7f9e5effd700 1 -- 172.21.15.47:0/2077308138 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64031050 con 0x5583f37c86a0 2024-05-11T12:30:35.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.374+0000 7f9e5effd700 1 -- 172.21.15.47:0/2077308138 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e6400b6b0 con 0x5583f37c86a0 2024-05-11T12:30:35.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.374+0000 7f9e5effd700 1 -- 172.21.15.47:0/2077308138 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e64003680 con 0x5583f37c86a0 2024-05-11T12:30:35.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.375+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/2077308138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 msgr2=0x5583f37cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.375+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9e64005040 tx=0x7f9e6400b5e0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.376+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/2077308138 shutdown_connections 2024-05-11T12:30:35.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.376+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37cef00 0x5583f37b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.376+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37c3b30 0x5583f37c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.376+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/2077308138 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37cce60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.376+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/2077308138 >> 172.21.15.47:0/2077308138 conn(0x5583f36efb70 msgr2=0x5583f37c2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:35.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.377+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/2077308138 shutdown_connections 2024-05-11T12:30:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.377+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/2077308138 wait complete. 2024-05-11T12:30:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.378+0000 7f9e85bf3b80 1 -- start start 2024-05-11T12:30:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.378+0000 7f9e85bf3b80 1 Processor -- start 2024-05-11T12:30:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.379+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.379+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.379+0000 7f9e85bf3b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37cef00 0x5583f37c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.379+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5800c460 con 0x5583f37cef00 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.379+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5800d8d0 con 0x5583f37c86a0 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.379+0000 7f9e85bf3b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5800cb30 con 0x5583f37c3b30 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.379+0000 7f9e85bf3b80 10 client.0 _ll_register_callbacks cb 0x5583f36ad950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.380+0000 7f9e798fa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37cef00 0x5583f37c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.380+0000 7f9e798fa700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37cef00 0x5583f37c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55468/0 (socket says 172.21.15.47:55468) 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.380+0000 7f9e798fa700 1 -- 172.21.15.47:0/1337060460 learned_addr learned my addr 172.21.15.47:0/1337060460 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.381+0000 7f9e788f8700 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.381+0000 7f9e790f9700 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.381+0000 7f9e798fa700 1 -- 172.21.15.47:0/1337060460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 msgr2=0x5583f37c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.381+0000 7f9e798fa700 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.381+0000 7f9e798fa700 1 -- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 msgr2=0x5583f37c0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.381+0000 7f9e798fa700 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37c0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.381+0000 7f9e798fa700 1 -- 172.21.15.47:0/1337060460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583f37d4710 con 0x5583f37cef00 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.382+0000 7f9e798fa700 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37cef00 0x5583f37c0f70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009460 tx=0x7f9e6c035840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30273c65ff405c2d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.382+0000 7f9e5cff9700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.382+0000 7f9e5cff9700 1 -- 172.21.15.47:0/1337060460 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c03a040 con 0x5583f37cef00 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.383+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e6400b6b0 con 0x5583f37cef00 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.383+0000 7f9e5cff9700 1 -- 172.21.15.47:0/1337060460 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9e6c035c60 con 0x5583f37cef00 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.383+0000 7f9e5cff9700 1 -- 172.21.15.47:0/1337060460 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9e6c009c20 con 0x5583f37cef00 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.383+0000 7f9e85bf3b80 10 client.15621 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.383+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5583f37d59f0 con 0x5583f37cef00 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.384+0000 7f9e5cff9700 1 -- 172.21.15.47:0/1337060460 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e6400b6b0 con 0x5583f37cef00 2024-05-11T12:30:35.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.384+0000 7f9e5cff9700 1 -- 172.21.15.47:0/1337060460 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f9e6c049470 con 0x5583f37cef00 2024-05-11T12:30:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.385+0000 7f9e85bf3b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.385+0000 7f9e85bf3b80 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.385+0000 7f9e85bf3b80 20 client.15621 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.385+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f3858ec0 0x5583f385b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.385+0000 7f9e85bf3b80 4 client.15621 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:30:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.385+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583f371d3c0 con 0x5583f3858ec0 2024-05-11T12:30:35.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.386+0000 7f9e790f9700 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f3858ec0 0x5583f385b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:35.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.389+0000 7f9e790f9700 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f3858ec0 0x5583f385b2b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0de908d04502f2 server_cookie=da1d66a2180842b2 in_seq=0 out_seq=0 2024-05-11T12:30:35.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.389+0000 7f9e5cff9700 10 client.15621 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:35.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.392+0000 7f9e5cff9700 1 -- 172.21.15.47:0/1337060460 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583f371d3c0 con 0x5583f3858ec0 2024-05-11T12:30:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.392+0000 7f9e5cff9700 10 client.15621 handle_command_reply: tid=0 2024-05-11T12:30:35.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.392+0000 7f9e85bf3b80 1 client.15621 shutdown 2024-05-11T12:30:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.393+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37cef00 msgr2=0x5583f37c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.393+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37cef00 0x5583f37c0f70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009460 tx=0x7f9e6c035840 comp rx=0 tx=0).stop 2024-05-11T12:30:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 shutdown_connections 2024-05-11T12:30:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5583f3858ec0 0x5583f385b2b0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583f37cef00 0x5583f37c0f70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5583f37c3b30 0x5583f37c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e85bf3b80 1 --2- 172.21.15.47:0/1337060460 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5583f37c86a0 0x5583f37c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 >> 172.21.15.47:0/1337060460 conn(0x5583f36efb70 msgr2=0x5583f37c2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e5cff9700 0 client.15621 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.394+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 shutdown_connections 2024-05-11T12:30:35.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.395+0000 7f9e85bf3b80 1 -- 172.21.15.47:0/1337060460 wait complete. 2024-05-11T12:30:35.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.398+0000 7f9e85bf3b80 20 client.15621 trim_cache size 0 max 16384 2024-05-11T12:30:35.402 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:30:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.404+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9e600b2a00 msgr2=0x7f9e600b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.404+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9e600b2a00 0x7f9e600b4ec0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9e7411f4c0 tx=0x7f9e64019040 comp rx=0 tx=0).stop 2024-05-11T12:30:35.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.404+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e7410aa90 msgr2=0x7f9e741ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:35.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.404+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e7410aa90 0x7f9e741ae790 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000d950 tx=0x7f9e7000dc60 comp rx=0 tx=0).stop 2024-05-11T12:30:35.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.407+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 shutdown_connections 2024-05-11T12:30:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.407+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f9e600b2a00 0x7f9e600b4ec0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.407+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9e7411cd30 0x7f9e741a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.407+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e7410aa90 0x7f9e741ae790 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.407+0000 7f9e7b35d700 1 --2- 172.21.15.47:0/396403318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9e74111d30 0x7f9e741aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:35.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.407+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 >> 172.21.15.47:0/396403318 conn(0x7f9e7406e6c0 msgr2=0x7f9e7411ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.407+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 shutdown_connections 2024-05-11T12:30:35.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:35.408+0000 7f9e7b35d700 1 -- 172.21.15.47:0/396403318 wait complete. 2024-05-11T12:30:35.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:35 smithi047 ceph-mon[30375]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 769 MiB/s wr, 1.41k op/s 2024-05-11T12:30:35.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:35 smithi064 ceph-mon[34537]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 769 MiB/s wr, 1.41k op/s 2024-05-11T12:30:35.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:35 smithi050 ceph-mon[34072]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 769 MiB/s wr, 1.41k op/s 2024-05-11T12:30:37.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:37 smithi050 ceph-mon[34072]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 756 MiB/s wr, 1.33k op/s 2024-05-11T12:30:37.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:37 smithi047 ceph-mon[30375]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 756 MiB/s wr, 1.33k op/s 2024-05-11T12:30:37.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:37 smithi064 ceph-mon[34537]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 756 MiB/s wr, 1.33k op/s 2024-05-11T12:30:38.154 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:30:38.154 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:30:39.065 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:30:39.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:39 smithi047 ceph-mon[30375]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 933 MiB/s wr, 1.65k op/s 2024-05-11T12:30:39.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:39 smithi050 ceph-mon[34072]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 933 MiB/s wr, 1.65k op/s 2024-05-11T12:30:39.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:39 smithi064 ceph-mon[34537]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 933 MiB/s wr, 1.65k op/s 2024-05-11T12:30:41.693 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:41 smithi047 ceph-mon[30375]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 783 MiB/s wr, 1.43k op/s 2024-05-11T12:30:41.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:41 smithi050 ceph-mon[34072]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 783 MiB/s wr, 1.43k op/s 2024-05-11T12:30:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:41 smithi064 ceph-mon[34537]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 783 MiB/s wr, 1.43k op/s 2024-05-11T12:30:43.589 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:43 smithi047 ceph-mon[30375]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 858 MiB/s wr, 1.54k op/s 2024-05-11T12:30:43.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:43 smithi050 ceph-mon[34072]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 858 MiB/s wr, 1.54k op/s 2024-05-11T12:30:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:43 smithi064 ceph-mon[34537]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 858 MiB/s wr, 1.54k op/s 2024-05-11T12:30:44.193 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:30:44.194 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:30:44.194 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "8feae9e9-b553-4940-8566-477c10203fff", 2024-05-11T12:30:44.194 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:30:44.194 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.873+0000 7ffb17fff700 1 -- 172.21.15.47:0/2613323990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 msgr2=0x7ffb181023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.873+0000 7ffb15ffb700 1 -- 172.21.15.47:0/2613323990 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08004230 con 0x7ffb18101fe0 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.873+0000 7ffb17fff700 1 --2- 172.21.15.47:0/2613323990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb181023c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009990 tx=0x7ffb08009ca0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.874+0000 7ffb17fff700 1 -- 172.21.15.47:0/2613323990 shutdown_connections 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.874+0000 7ffb17fff700 1 --2- 172.21.15.47:0/2613323990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb18108850 0x7ffb18110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.874+0000 7ffb17fff700 1 --2- 172.21.15.47:0/2613323990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb181023c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.874+0000 7ffb17fff700 1 --2- 172.21.15.47:0/2613323990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb18102990 0x7ffb18108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.874+0000 7ffb17fff700 1 -- 172.21.15.47:0/2613323990 >> 172.21.15.47:0/2613323990 conn(0x7ffb18077700 msgr2=0x7ffb18077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.875+0000 7ffb17fff700 1 -- 172.21.15.47:0/2613323990 shutdown_connections 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.875+0000 7ffb17fff700 1 -- 172.21.15.47:0/2613323990 wait complete. 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.877+0000 7ffb17fff700 1 Processor -- start 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.877+0000 7ffb17fff700 1 -- start start 2024-05-11T12:30:44.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.877+0000 7ffb17fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb1810ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.877+0000 7ffb17fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb18102990 0x7ffb180733f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb17fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb18108850 0x7ffb18073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb1810ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb1810ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52746/0 (socket says 172.21.15.47:52746) 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb17fff700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1806eb30 con 0x7ffb18101fe0 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb17fff700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1806e9b0 con 0x7ffb18102990 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb17fff700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1806ecb0 con 0x7ffb18108850 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb16ffd700 1 -- 172.21.15.47:0/327064439 learned_addr learned my addr 172.21.15.47:0/327064439 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb16ffd700 1 -- 172.21.15.47:0/327064439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb18108850 msgr2=0x7ffb18073930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb18108850 0x7ffb18073930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb16ffd700 1 -- 172.21.15.47:0/327064439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb18102990 msgr2=0x7ffb180733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.878+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb18102990 0x7ffb180733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.879+0000 7ffb16ffd700 1 -- 172.21.15.47:0/327064439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb18073ef0 con 0x7ffb18101fe0 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.890+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb1810ff70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffb080057c0 tx=0x7ffb08005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:44.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.893+0000 7ffafffff700 1 -- 172.21.15.47:0/327064439 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08027070 con 0x7ffb18101fe0 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.893+0000 7ffafffff700 1 -- 172.21.15.47:0/327064439 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb08022470 con 0x7ffb18101fe0 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.893+0000 7ffafffff700 1 -- 172.21.15.47:0/327064439 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08017650 con 0x7ffb18101fe0 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.893+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb18074180 con 0x7ffb18101fe0 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.893+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1810dc00 con 0x7ffb18101fe0 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.914+0000 7ffafffff700 1 -- 172.21.15.47:0/327064439 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ffb0802a070 con 0x7ffb18101fe0 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.914+0000 7ffb27d98b80 1 Processor -- start 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.915+0000 7ffb27d98b80 1 -- start start 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.915+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268db6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.915+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.915+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 0x5649268e0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.915+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56492682d8e0 con 0x5649268db300 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.915+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5649267cc840 con 0x5649268dbc20 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.915+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5649268298a0 con 0x5649268dfdd0 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.916+0000 7ffafffff700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ffb000b2a00 0x7ffb000b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.916+0000 7ffafffff700 1 -- 172.21.15.47:0/327064439 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ffb08022620 con 0x7ffb18101fe0 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.918+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.918+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268df890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:45672/0 (socket says 172.21.15.47:45672) 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.918+0000 7ffb16ffd700 1 -- 172.21.15.47:0/55607555 learned_addr learned my addr 172.21.15.47:0/55607555 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.919+0000 7ffb16ffd700 1 -- 172.21.15.47:0/55607555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 msgr2=0x5649268e0210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.919+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/55607555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 0x5649268e0210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.919+0000 7ffb16ffd700 1 -- 172.21.15.47:0/55607555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 msgr2=0x5649268db6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.919+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/55607555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268db6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.919+0000 7ffb16ffd700 1 -- 172.21.15.47:0/55607555 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56492677a950 con 0x5649268dbc20 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.919+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/55607555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268df890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb080058b0 tx=0x7ffb080aae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e4ec8a213739467 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.920+0000 7ffafd7fa700 1 -- 172.21.15.47:0/55607555 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080ab700 con 0x5649268dbc20 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.920+0000 7ffafd7fa700 1 -- 172.21.15.47:0/55607555 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb080afab0 con 0x5649268dbc20 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.921+0000 7ffb177fe700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ffb000b2a00 0x7ffb000b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.921+0000 7ffb27d98b80 1 -- 172.21.15.47:0/55607555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 msgr2=0x5649268df890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.921+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/55607555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268df890 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ffb080058b0 tx=0x7ffb080aae70 comp rx=0 tx=0).stop 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.926+0000 7ffb27d98b80 1 -- 172.21.15.47:0/55607555 shutdown_connections 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.926+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/55607555 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 0x5649268e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.926+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/55607555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268db6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.926+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/55607555 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268df890 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.926+0000 7ffb27d98b80 1 -- 172.21.15.47:0/55607555 >> 172.21.15.47:0/55607555 conn(0x564926807ee0 msgr2=0x5649268d5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:44.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.934+0000 7ffb27d98b80 1 -- 172.21.15.47:0/55607555 shutdown_connections 2024-05-11T12:30:44.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.936+0000 7ffb177fe700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ffb000b2a00 0x7ffb000b4ec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009690 tx=0x7ffb100095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:44.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.937+0000 7ffb27d98b80 1 -- 172.21.15.47:0/55607555 wait complete. 2024-05-11T12:30:44.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.938+0000 7ffb27d98b80 1 -- start start 2024-05-11T12:30:44.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.938+0000 7ffb27d98b80 1 Processor -- start 2024-05-11T12:30:44.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.940+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.940+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.940+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 0x5649268e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.940+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5649268e7df0 con 0x5649268db300 2024-05-11T12:30:44.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.940+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564926793cb0 con 0x5649268dbc20 2024-05-11T12:30:44.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.940+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5649268e7f70 con 0x5649268dfdd0 2024-05-11T12:30:44.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.940+0000 7ffb27d98b80 10 client.0 _ll_register_callbacks cb 0x5649267c59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:44.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.941+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.941+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268e72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52766/0 (socket says 172.21.15.47:52766) 2024-05-11T12:30:44.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.941+0000 7ffb16ffd700 1 -- 172.21.15.47:0/4152757761 learned_addr learned my addr 172.21.15.47:0/4152757761 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:44.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.944+0000 7ffb177fe700 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 0x5649268e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.944+0000 7ffb16ffd700 1 -- 172.21.15.47:0/4152757761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 msgr2=0x5649268e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.944+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 0x5649268e4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.944+0000 7ffb16ffd700 1 -- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 msgr2=0x5649268e46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.944+0000 7ffb167fc700 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.944+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.944+0000 7ffb16ffd700 1 -- 172.21.15.47:0/4152757761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649267c9710 con 0x5649268db300 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.947+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268e72c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ffb080ab690 tx=0x7ffb080abec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7d3db2d0c9300 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.949+0000 7ffafaff5700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.949+0000 7ffafaff5700 1 -- 172.21.15.47:0/4152757761 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080afab0 con 0x5649268db300 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.949+0000 7ffafaff5700 1 -- 172.21.15.47:0/4152757761 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ffb08005320 con 0x5649268db300 2024-05-11T12:30:44.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.949+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56492683b2c0 con 0x5649268db300 2024-05-11T12:30:44.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.954+0000 7ffafaff5700 1 -- 172.21.15.47:0/4152757761 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffb080bb050 con 0x5649268db300 2024-05-11T12:30:44.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.955+0000 7ffb27d98b80 10 client.15630 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:44.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.955+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5649268e7df0 con 0x5649268db300 2024-05-11T12:30:44.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffafaff5700 1 -- 172.21.15.47:0/4152757761 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56492683b2c0 con 0x5649268db300 2024-05-11T12:30:44.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffafaff5700 1 -- 172.21.15.47:0/4152757761 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ffb080c0bd0 con 0x5649268db300 2024-05-11T12:30:44.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffb27d98b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffb27d98b80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffb27d98b80 20 client.15630 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5649268f4d10 0x5649268f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffb27d98b80 4 client.15630 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.959+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564926835360 con 0x5649268f4d10 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.963+0000 7ffb167fc700 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5649268f4d10 0x5649268f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.974+0000 7ffb167fc700 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5649268f4d10 0x5649268f7120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47843d74d7adb121 server_cookie=ba701786c073ae74 in_seq=0 out_seq=0 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.974+0000 7ffafaff5700 10 client.15630 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.984+0000 7ffafaff5700 1 -- 172.21.15.47:0/4152757761 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564926835360 con 0x5649268f4d10 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.984+0000 7ffafaff5700 10 client.15630 handle_command_reply: tid=0 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.985+0000 7ffb27d98b80 1 client.15630 shutdown 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.986+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 msgr2=0x5649268e72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.986+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268e72c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ffb080ab690 tx=0x7ffb080abec0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 shutdown_connections 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5649268f4d10 0x5649268f7120 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268dfdd0 0x5649268e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268db300 0x5649268e72c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4152757761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 >> 172.21.15.47:0/4152757761 conn(0x564926807ee0 msgr2=0x5649268da110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffafaff5700 0 client.15630 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.988+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 shutdown_connections 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.990+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4152757761 wait complete. 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:43.992+0000 7ffb27d98b80 20 client.15630 trim_cache size 0 max 16384 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.055+0000 7ffb27d98b80 1 Processor -- start 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.056+0000 7ffb27d98b80 1 -- start start 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.056+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268f0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649268fca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181c3510 con 0x5649268dfdd0 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268f0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268f0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40848/0 (socket says 172.21.15.47:40848) 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb16ffd700 1 -- 172.21.15.47:0/4146932507 learned_addr learned my addr 172.21.15.47:0/4146932507 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb167fc700 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4146932507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181c5d70 con 0x5649268dbc20 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.057+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4146932507 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181c3b60 con 0x5649268db300 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.060+0000 7ffb177fe700 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649268fca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.062+0000 7ffb167fc700 1 -- 172.21.15.47:0/4146932507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 msgr2=0x5649268f0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.062+0000 7ffb167fc700 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268f0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.062+0000 7ffb167fc700 1 -- 172.21.15.47:0/4146932507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 msgr2=0x5649268fca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.062+0000 7ffb167fc700 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649268fca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.062+0000 7ffb167fc700 1 -- 172.21.15.47:0/4146932507 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649268e5590 con 0x5649268dbc20 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.068+0000 7ffb167fc700 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5649268d6880 tx=0x7ffb0c00a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5eb7783543c426e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.069+0000 7ffafcff9700 1 -- 172.21.15.47:0/4146932507 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c0170f0 con 0x5649268dbc20 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.069+0000 7ffafcff9700 1 -- 172.21.15.47:0/4146932507 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb0c01b030 con 0x5649268dbc20 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.069+0000 7ffafcff9700 1 -- 172.21.15.47:0/4146932507 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c015690 con 0x5649268dbc20 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.071+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4146932507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 msgr2=0x5649268e3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.071+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x5649268d6880 tx=0x7ffb0c00a940 comp rx=0 tx=0).stop 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.072+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4146932507 shutdown_connections 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.072+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649268fca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.072+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268f0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.072+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/4146932507 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e3830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.072+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4146932507 >> 172.21.15.47:0/4146932507 conn(0x564926807ee0 msgr2=0x5649268da780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.080+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4146932507 shutdown_connections 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.081+0000 7ffb27d98b80 1 -- 172.21.15.47:0/4146932507 wait complete. 2024-05-11T12:30:44.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.081+0000 7ffb27d98b80 1 -- start start 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.082+0000 7ffb27d98b80 1 Processor -- start 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.083+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.083+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.083+0000 7ffb27d98b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649269575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.084+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181c4230 con 0x5649268dfdd0 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.084+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181c56a0 con 0x5649268dbc20 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.084+0000 7ffb27d98b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ffb181c4900 con 0x5649268db300 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.084+0000 7ffb27d98b80 10 client.0 _ll_register_callbacks cb 0x5649267c59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.087+0000 7ffb167fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.087+0000 7ffb177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649269575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.087+0000 7ffb16ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.087+0000 7ffb177fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649269575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52784/0 (socket says 172.21.15.47:52784) 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.087+0000 7ffb177fe700 1 -- 172.21.15.47:0/1586372055 learned_addr learned my addr 172.21.15.47:0/1586372055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.093+0000 7ffb16ffd700 1 -- 172.21.15.47:0/1586372055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 msgr2=0x5649268e9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:44.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.093+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.093+0000 7ffb16ffd700 1 -- 172.21.15.47:0/1586372055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 msgr2=0x5649269575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.093+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649269575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.093+0000 7ffb16ffd700 1 -- 172.21.15.47:0/1586372055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564926957dc0 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.096+0000 7ffb16ffd700 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268e99b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb080aa570 tx=0x7ffb080b43b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df2064d792fb23cd server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.096+0000 7ffafaff5700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.097+0000 7ffafaff5700 1 -- 172.21.15.47:0/1586372055 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb080afab0 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.097+0000 7ffafaff5700 1 -- 172.21.15.47:0/1586372055 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ffb080c06f0 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.097+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb0c01b030 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.098+0000 7ffafaff5700 1 -- 172.21.15.47:0/1586372055 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ffb080b44d0 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.098+0000 7ffb27d98b80 10 client.25264 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.098+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564926957dc0 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.099+0000 7ffafaff5700 1 -- 172.21.15.47:0/1586372055 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb080ca070 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.099+0000 7ffafaff5700 1 -- 172.21.15.47:0/1586372055 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ffb080a7070 con 0x5649268db300 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.099+0000 7ffb27d98b80 10 client.25264 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:44.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.099+0000 7ffb27d98b80 10 client.25264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.099+0000 7ffb27d98b80 20 client.25264 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.100+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564926960390 0x564926962780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.100+0000 7ffb27d98b80 4 client.25264 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.100+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ffb0c01b030 con 0x564926960390 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.103+0000 7ffb167fc700 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564926960390 0x564926962780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.112+0000 7ffb167fc700 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564926960390 0x564926962780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc99676a444718c server_cookie=9a11f27bb847f56d in_seq=0 out_seq=0 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.112+0000 7ffafaff5700 10 client.25264 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.117+0000 7ffafaff5700 1 -- 172.21.15.47:0/1586372055 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ffb0c01b030 con 0x564926960390 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffafaff5700 10 client.25264 handle_command_reply: tid=0 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 client.25264 shutdown 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 msgr2=0x5649268e99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268e99b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb080aa570 tx=0x7ffb080b43b0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 shutdown_connections 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564926960390 0x564926962780 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5649268dfdd0 0x5649269575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5649268db300 0x5649268e99b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 --2- 172.21.15.47:0/1586372055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5649268dbc20 0x5649268e9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 >> 172.21.15.47:0/1586372055 conn(0x564926807ee0 msgr2=0x5649268de790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.118+0000 7ffafaff5700 0 client.25264 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.119+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 shutdown_connections 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.119+0000 7ffb27d98b80 1 -- 172.21.15.47:0/1586372055 wait complete. 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.119+0000 7ffb27d98b80 20 client.25264 trim_cache size 0 max 16384 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.121+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ffb000b2a00 msgr2=0x7ffb000b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.121+0000 7ffb17fff700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ffb000b2a00 0x7ffb000b4ec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009690 tx=0x7ffb100095e0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.121+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 msgr2=0x7ffb1810ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.121+0000 7ffb17fff700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb1810ff70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffb080057c0 tx=0x7ffb08005990 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.122+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 shutdown_connections 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.122+0000 7ffb17fff700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ffb18108850 0x7ffb18073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.122+0000 7ffb17fff700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffb18101fe0 0x7ffb1810ff70 secure :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffb080057c0 tx=0x7ffb08005990 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.122+0000 7ffb17fff700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ffb000b2a00 0x7ffb000b4ec0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.122+0000 7ffb17fff700 1 --2- 172.21.15.47:0/327064439 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb18102990 0x7ffb180733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:44.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.122+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 >> 172.21.15.47:0/327064439 conn(0x7ffb18077700 msgr2=0x7ffb18100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:44.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.125+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 shutdown_connections 2024-05-11T12:30:44.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:44.125+0000 7ffb17fff700 1 -- 172.21.15.47:0/327064439 wait complete. 2024-05-11T12:30:45.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:45 smithi047 ceph-mon[30375]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.40k op/s 2024-05-11T12:30:45.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:45 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:30:45.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:45 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:45.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:45 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:30:45.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:45 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:45.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:45 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:30:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:45 smithi064 ceph-mon[34537]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.40k op/s 2024-05-11T12:30:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:45 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:30:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:45 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:45 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:30:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:45 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:45 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:30:45.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:30:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:45 smithi050 ceph-mon[34072]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 798 MiB/s wr, 1.40k op/s 2024-05-11T12:30:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:45 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:30:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:45 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:45 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:30:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:45 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:30:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:45 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:30:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:46 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:46 smithi047 ceph-mon[30375]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-11T12:30:46.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:46 smithi047 ceph-mon[30375]: Unable to set osd_memory_target on smithi064 to 184793190: error parsing value: Value '184793190' is below minimum 939524096 2024-05-11T12:30:46.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:46 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:46 smithi064 ceph-mon[34537]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-11T12:30:46.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:46 smithi064 ceph-mon[34537]: Unable to set osd_memory_target on smithi064 to 184793190: error parsing value: Value '184793190' is below minimum 939524096 2024-05-11T12:30:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:46 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-11T12:30:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:46 smithi050 ceph-mon[34072]: Adjusting osd_memory_target on smithi064 to 176.2M 2024-05-11T12:30:46.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:46 smithi050 ceph-mon[34072]: Unable to set osd_memory_target on smithi064 to 184793190: error parsing value: Value '184793190' is below minimum 939524096 2024-05-11T12:30:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:30:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:47 smithi064 ceph-mon[34537]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 755 MiB/s wr, 1.37k op/s 2024-05-11T12:30:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:30:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:30:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:47 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:30:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:47 smithi050 ceph-mon[34072]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 755 MiB/s wr, 1.37k op/s 2024-05-11T12:30:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:30:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:30:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:47 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:30:47.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:47 smithi047 ceph-mon[30375]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 755 MiB/s wr, 1.37k op/s 2024-05-11T12:30:47.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:30:47.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:30:47.957 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:47 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:30:49.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:49 smithi064 ceph-mon[34537]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 917 MiB/s wr, 1.75k op/s 2024-05-11T12:30:49.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:49 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:30:49.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:49 smithi050 ceph-mon[34072]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 917 MiB/s wr, 1.75k op/s 2024-05-11T12:30:49.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:49 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:30:50.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:49 smithi047 ceph-mon[30375]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 917 MiB/s wr, 1.75k op/s 2024-05-11T12:30:50.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:49 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:30:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.561+0000 7fefedb4a700 1 -- 172.21.15.47:0/2299196541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8105f80 msgr2=0x7fefe8106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.561+0000 7fefedb4a700 1 --2- 172.21.15.47:0/2299196541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8105f80 0x7fefe8106410 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fefe000b0a0 tx=0x7fefe000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.563+0000 7fefedb4a700 1 -- 172.21.15.47:0/2299196541 shutdown_connections 2024-05-11T12:30:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.563+0000 7fefedb4a700 1 --2- 172.21.15.47:0/2299196541 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8105f80 0x7fefe8106410 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.563+0000 7fefedb4a700 1 --2- 172.21.15.47:0/2299196541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe8108470 0x7fefe806baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.563+0000 7fefedb4a700 1 --2- 172.21.15.47:0/2299196541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefe806c030 0x7fefe806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.563+0000 7fefedb4a700 1 -- 172.21.15.47:0/2299196541 >> 172.21.15.47:0/2299196541 conn(0x7fefe80793c0 msgr2=0x7fefe80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:51.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.565+0000 7fefedb4a700 1 -- 172.21.15.47:0/2299196541 shutdown_connections 2024-05-11T12:30:51.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.566+0000 7fefedb4a700 1 -- 172.21.15.47:0/2299196541 wait complete. 2024-05-11T12:30:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.567+0000 7fefedb4a700 1 Processor -- start 2024-05-11T12:30:51.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.567+0000 7fefedb4a700 1 -- start start 2024-05-11T12:30:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.568+0000 7fefedb4a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe806c030 0x7fefe81168b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.568+0000 7fefedb4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefe8105f80 0x7fefe8110910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.568+0000 7fefedb4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8108470 0x7fefe8110e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.568+0000 7fefedb4a700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8109340 con 0x7fefe8105f80 2024-05-11T12:30:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.568+0000 7fefedb4a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefe81091c0 con 0x7fefe806c030 2024-05-11T12:30:51.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.568+0000 7fefedb4a700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefe81094c0 con 0x7fefe8108470 2024-05-11T12:30:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.572+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefe8105f80 0x7fefe8110910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.572+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefe8105f80 0x7fefe8110910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36076/0 (socket says 172.21.15.47:36076) 2024-05-11T12:30:51.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.572+0000 7fefe6ffd700 1 -- 172.21.15.47:0/190079988 learned_addr learned my addr 172.21.15.47:0/190079988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:51.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.574+0000 7fefe7fff700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8108470 0x7fefe8110e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.574+0000 7fefe77fe700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe806c030 0x7fefe81168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.582+0000 7fefe7fff700 1 -- 172.21.15.47:0/190079988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe806c030 msgr2=0x7fefe81168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.582+0000 7fefe7fff700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe806c030 0x7fefe81168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.582+0000 7fefe7fff700 1 -- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefe8105f80 msgr2=0x7fefe8110910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.582+0000 7fefe7fff700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefe8105f80 0x7fefe8110910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.582+0000 7fefe7fff700 1 -- 172.21.15.47:0/190079988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe8111490 con 0x7fefe8108470 2024-05-11T12:30:51.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.584+0000 7fefe7fff700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8108470 0x7fefe8110e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefe000b070 tx=0x7fefe0004d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.588+0000 7fefe4ff9700 1 -- 172.21.15.47:0/190079988 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0021070 con 0x7fefe8108470 2024-05-11T12:30:51.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.588+0000 7fefe4ff9700 1 -- 172.21.15.47:0/190079988 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe0007b80 con 0x7fefe8108470 2024-05-11T12:30:51.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.588+0000 7fefe4ff9700 1 -- 172.21.15.47:0/190079988 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0005040 con 0x7fefe8108470 2024-05-11T12:30:51.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.589+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe8111720 con 0x7fefe8108470 2024-05-11T12:30:51.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.589+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe81b5400 con 0x7fefe8108470 2024-05-11T12:30:51.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.595+0000 7feff83e0b80 1 Processor -- start 2024-05-11T12:30:51.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.599+0000 7feff83e0b80 1 -- start start 2024-05-11T12:30:51.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.599+0000 7fefe4ff9700 1 -- 172.21.15.47:0/190079988 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fefe0024070 con 0x7fefe8108470 2024-05-11T12:30:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.600+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519872bad0 0x56519872beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.600+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x565198726520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.600+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198732890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.600+0000 7feff83e0b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x565198678860 con 0x5651987260a0 2024-05-11T12:30:51.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.600+0000 7feff83e0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5651986177c0 con 0x565198726a60 2024-05-11T12:30:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.600+0000 7feff83e0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565198674820 con 0x56519872bad0 2024-05-11T12:30:51.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.602+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519872bad0 0x56519872beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.604+0000 7fefe4ff9700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefd00b2a90 0x7fefd00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.604+0000 7fefe77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x565198726520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.604+0000 7fefe77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x565198726520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36084/0 (socket says 172.21.15.47:36084) 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.604+0000 7fefe77fe700 1 -- 172.21.15.47:0/248944015 learned_addr learned my addr 172.21.15.47:0/248944015 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.604+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198732890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.604+0000 7fefe4ff9700 1 -- 172.21.15.47:0/190079988 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fefe000f020 con 0x7fefe8108470 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.605+0000 7fefe7fff700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefd00b2a90 0x7fefd00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.605+0000 7fefe6ffd700 1 -- 172.21.15.47:0/248944015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 msgr2=0x565198732890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.605+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198732890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.605+0000 7fefe6ffd700 1 -- 172.21.15.47:0/248944015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 msgr2=0x565198726520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.605+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x565198726520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.605+0000 7fefe6ffd700 1 -- 172.21.15.47:0/248944015 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651985c58d0 con 0x56519872bad0 2024-05-11T12:30:51.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.606+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519872bad0 0x56519872beb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefe81120f0 tx=0x7fefdc00b780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb2a934cc3d76a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:51.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.608+0000 7fefe7fff700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefd00b2a90 0x7fefd00b4f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fefe001db20 tx=0x7fefe0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.611+0000 7fefcdffb700 1 -- 172.21.15.47:0/248944015 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc004a30 con 0x56519872bad0 2024-05-11T12:30:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.611+0000 7fefcdffb700 1 -- 172.21.15.47:0/248944015 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefdc0054d0 con 0x56519872bad0 2024-05-11T12:30:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.611+0000 7fefcdffb700 1 -- 172.21.15.47:0/248944015 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc010730 con 0x56519872bad0 2024-05-11T12:30:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.613+0000 7feff83e0b80 1 -- 172.21.15.47:0/248944015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519872bad0 msgr2=0x56519872beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.613+0000 7feff83e0b80 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519872bad0 0x56519872beb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fefe81120f0 tx=0x7fefdc00b780 comp rx=0 tx=0).stop 2024-05-11T12:30:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.615+0000 7feff83e0b80 1 -- 172.21.15.47:0/248944015 shutdown_connections 2024-05-11T12:30:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.615+0000 7feff83e0b80 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198732890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.615+0000 7feff83e0b80 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56519872bad0 0x56519872beb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.615+0000 7feff83e0b80 1 --2- 172.21.15.47:0/248944015 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x565198726520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.615+0000 7feff83e0b80 1 -- 172.21.15.47:0/248944015 >> 172.21.15.47:0/248944015 conn(0x565198652e60 msgr2=0x565198720990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.619+0000 7feff83e0b80 1 -- 172.21.15.47:0/248944015 shutdown_connections 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.619+0000 7feff83e0b80 1 -- 172.21.15.47:0/248944015 wait complete. 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.620+0000 7feff83e0b80 1 -- start start 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.620+0000 7feff83e0b80 1 Processor -- start 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.621+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198732530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.621+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x56519872f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.621+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519872bad0 0x56519872fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.621+0000 7feff83e0b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x565198732e90 con 0x56519872bad0 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.621+0000 7feff83e0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5651985dec30 con 0x565198726a60 2024-05-11T12:30:51.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.621+0000 7feff83e0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x565198733010 con 0x5651987260a0 2024-05-11T12:30:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.621+0000 7feff83e0b80 10 client.0 _ll_register_callbacks cb 0x565198610920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.622+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519872bad0 0x56519872fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.622+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519872bad0 0x56519872fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36098/0 (socket says 172.21.15.47:36098) 2024-05-11T12:30:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.622+0000 7fefe6ffd700 1 -- 172.21.15.47:0/2892528364 learned_addr learned my addr 172.21.15.47:0/2892528364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.626+0000 7fefe77fe700 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x56519872f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.626+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198732530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.626+0000 7fefe6ffd700 1 -- 172.21.15.47:0/2892528364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 msgr2=0x565198732530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.626+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198732530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.626+0000 7fefe6ffd700 1 -- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 msgr2=0x56519872f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.626+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x56519872f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.626+0000 7fefe6ffd700 1 -- 172.21.15.47:0/2892528364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565198614690 con 0x56519872bad0 2024-05-11T12:30:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.630+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519872bad0 0x56519872fe10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00b880 tx=0x7fefdc012f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c0e5c7b3582e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.631+0000 7fefc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:30:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.631+0000 7fefc6ffd700 1 -- 172.21.15.47:0/2892528364 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc027040 con 0x56519872bad0 2024-05-11T12:30:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.631+0000 7fefc6ffd700 1 -- 172.21.15.47:0/2892528364 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fefdc004b70 con 0x56519872bad0 2024-05-11T12:30:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.631+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefdc0054d0 con 0x56519872bad0 2024-05-11T12:30:51.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.632+0000 7fefc6ffd700 1 -- 172.21.15.47:0/2892528364 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fefdc032020 con 0x56519872bad0 2024-05-11T12:30:51.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.634+0000 7feff83e0b80 10 client.15648 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:51.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.634+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56519874e730 con 0x56519872bad0 2024-05-11T12:30:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.635+0000 7fefc6ffd700 1 -- 172.21.15.47:0/2892528364 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefdc0054d0 con 0x56519872bad0 2024-05-11T12:30:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.637+0000 7fefc6ffd700 1 -- 172.21.15.47:0/2892528364 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fefdc0304b0 con 0x56519872bad0 2024-05-11T12:30:51.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.637+0000 7feff83e0b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:51.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.637+0000 7feff83e0b80 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:51.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.637+0000 7feff83e0b80 20 client.15648 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:51.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.637+0000 7feff83e0b80 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x565198750f60 0x565198753380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.637+0000 7feff83e0b80 4 client.15648 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:30:51.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.637+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565198686240 con 0x565198750f60 2024-05-11T12:30:51.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.639+0000 7fefe77fe700 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x565198750f60 0x565198753380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.641+0000 7fefe77fe700 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x565198750f60 0x565198753380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f2b954a5039a3d server_cookie=85b343a5e1719306 in_seq=0 out_seq=0 2024-05-11T12:30:51.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.644+0000 7fefc6ffd700 10 client.15648 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:51.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.645+0000 7fefc6ffd700 1 -- 172.21.15.47:0/2892528364 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x565198686240 con 0x565198750f60 2024-05-11T12:30:51.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.645+0000 7fefc6ffd700 10 client.15648 handle_command_reply: tid=0 2024-05-11T12:30:51.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.645+0000 7feff83e0b80 1 client.15648 shutdown 2024-05-11T12:30:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519872bad0 msgr2=0x56519872fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519872bad0 0x56519872fe10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00b880 tx=0x7fefdc012f60 comp rx=0 tx=0).stop 2024-05-11T12:30:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 shutdown_connections 2024-05-11T12:30:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x565198750f60 0x565198753380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519872bad0 0x56519872fe10 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fefdc00b880 tx=0x7fefdc012f60 comp rx=0 tx=0).stop 2024-05-11T12:30:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198732530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 --2- 172.21.15.47:0/2892528364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x56519872f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 >> 172.21.15.47:0/2892528364 conn(0x565198652e60 msgr2=0x56519871b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7fefc6ffd700 0 client.15648 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 shutdown_connections 2024-05-11T12:30:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.654+0000 7feff83e0b80 1 -- 172.21.15.47:0/2892528364 wait complete. 2024-05-11T12:30:51.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.655+0000 7feff83e0b80 20 client.15648 trim_cache size 0 max 16384 2024-05-11T12:30:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.707+0000 7feff83e0b80 1 Processor -- start 2024-05-11T12:30:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.708+0000 7feff83e0b80 1 -- start start 2024-05-11T12:30:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.708+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x5651987257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565198726a60 0x565198729550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56519873bee0 0x565198729a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7feff83e0b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8072520 con 0x5651987260a0 2024-05-11T12:30:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7feff83e0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8074cf0 con 0x56519873bee0 2024-05-11T12:30:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7fefe77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565198726a60 0x565198729550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7fefe77fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565198726a60 0x565198729550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:43330/0 (socket says 172.21.15.47:43330) 2024-05-11T12:30:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7fefe77fe700 1 -- 172.21.15.47:0/4070020037 learned_addr learned my addr 172.21.15.47:0/4070020037 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:51.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.709+0000 7feff83e0b80 1 -- 172.21.15.47:0/4070020037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8072ae0 con 0x565198726a60 2024-05-11T12:30:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.712+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x5651987257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.712+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56519873bee0 0x565198729a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.712+0000 7fefe77fe700 1 -- 172.21.15.47:0/4070020037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56519873bee0 msgr2=0x565198729a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.712+0000 7fefe77fe700 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56519873bee0 0x565198729a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.712+0000 7fefe77fe700 1 -- 172.21.15.47:0/4070020037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 msgr2=0x5651987257a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.712+0000 7fefe77fe700 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x5651987257a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.712+0000 7fefe77fe700 1 -- 172.21.15.47:0/4070020037 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519871d590 con 0x565198726a60 2024-05-11T12:30:51.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.715+0000 7fefe77fe700 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565198726a60 0x565198729550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000c00 tx=0x7fefd8013740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=451a14f652453e96 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:51.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.719+0000 7fefcd7fa700 1 -- 172.21.15.47:0/4070020037 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8013c20 con 0x565198726a60 2024-05-11T12:30:51.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.719+0000 7fefcd7fa700 1 -- 172.21.15.47:0/4070020037 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd8004500 con 0x565198726a60 2024-05-11T12:30:51.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.719+0000 7fefcd7fa700 1 -- 172.21.15.47:0/4070020037 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd8003d30 con 0x565198726a60 2024-05-11T12:30:51.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.721+0000 7feff83e0b80 1 -- 172.21.15.47:0/4070020037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565198726a60 msgr2=0x565198729550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.721+0000 7feff83e0b80 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565198726a60 0x565198729550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000c00 tx=0x7fefd8013740 comp rx=0 tx=0).stop 2024-05-11T12:30:51.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.725+0000 7feff83e0b80 1 -- 172.21.15.47:0/4070020037 shutdown_connections 2024-05-11T12:30:51.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.725+0000 7feff83e0b80 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56519873bee0 0x565198729a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.725+0000 7feff83e0b80 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5651987260a0 0x5651987257a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.725+0000 7feff83e0b80 1 --2- 172.21.15.47:0/4070020037 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x565198726a60 0x565198729550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.725+0000 7feff83e0b80 1 -- 172.21.15.47:0/4070020037 >> 172.21.15.47:0/4070020037 conn(0x565198652e60 msgr2=0x565198725090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.727+0000 7feff83e0b80 1 -- 172.21.15.47:0/4070020037 shutdown_connections 2024-05-11T12:30:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.727+0000 7feff83e0b80 1 -- 172.21.15.47:0/4070020037 wait complete. 2024-05-11T12:30:51.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.727+0000 7feff83e0b80 1 -- start start 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 1 Processor -- start 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198758770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198758cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519873bee0 0x5651987591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fefe80731b0 con 0x56519873bee0 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8074620 con 0x565198726a60 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8073880 con 0x5651987260a0 2024-05-11T12:30:51.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.728+0000 7feff83e0b80 10 client.0 _ll_register_callbacks cb 0x565198610920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:30:51.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.732+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198758770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.732+0000 7fefe6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198758770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:43332/0 (socket says 172.21.15.47:43332) 2024-05-11T12:30:51.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.732+0000 7fefe6ffd700 1 -- 172.21.15.47:0/232763183 learned_addr learned my addr 172.21.15.47:0/232763183 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:30:51.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.732+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519873bee0 0x5651987591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.732+0000 7fefe77fe700 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198758cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.738+0000 7fefe77fe700 1 -- 172.21.15.47:0/232763183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 msgr2=0x565198758770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.738+0000 7fefe77fe700 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198758770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.738+0000 7fefe77fe700 1 -- 172.21.15.47:0/232763183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519873bee0 msgr2=0x5651987591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:30:51.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.738+0000 7fefe77fe700 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519873bee0 0x5651987591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.738+0000 7fefe77fe700 1 -- 172.21.15.47:0/232763183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651987baae0 con 0x565198726a60 2024-05-11T12:30:51.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.738+0000 7fefe77fe700 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198758cb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefd8020040 tx=0x7fefd8007970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f2cf9aa86c30e10 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:30:51.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.739+0000 7fefc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:30:51.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.741+0000 7fefc6ffd700 1 -- 172.21.15.47:0/232763183 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd802b070 con 0x565198726a60 2024-05-11T12:30:51.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.741+0000 7fefc6ffd700 1 -- 172.21.15.47:0/232763183 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fefd8004050 con 0x565198726a60 2024-05-11T12:30:51.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.741+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd8004500 con 0x565198726a60 2024-05-11T12:30:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.745+0000 7fefc6ffd700 1 -- 172.21.15.47:0/232763183 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fefd8037050 con 0x565198726a60 2024-05-11T12:30:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.745+0000 7feff83e0b80 10 client.25235 fetch_fsmap learned FSMap version 20 2024-05-11T12:30:51.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.745+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5651987bbdd0 con 0x565198726a60 2024-05-11T12:30:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.748+0000 7fefc6ffd700 1 -- 172.21.15.47:0/232763183 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd8004500 con 0x565198726a60 2024-05-11T12:30:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.748+0000 7fefc6ffd700 1 -- 172.21.15.47:0/232763183 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fefd8011440 con 0x565198726a60 2024-05-11T12:30:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.750+0000 7feff83e0b80 10 client.25235 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:30:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.750+0000 7feff83e0b80 10 client.25235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:30:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.751+0000 7feff83e0b80 20 client.25235 populate_metadata read hostname 'smithi047' 2024-05-11T12:30:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.751+0000 7feff83e0b80 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5651987bc3e0 0x5651987be800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:30:51.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.751+0000 7feff83e0b80 4 client.25235 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:30:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.751+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651986802e0 con 0x5651987bc3e0 2024-05-11T12:30:51.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.752+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5651987bc3e0 0x5651987be800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:30:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.759+0000 7fefe6ffd700 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5651987bc3e0 0x5651987be800 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11f1578e2f914d1a server_cookie=e93f33cfc3cbe9de in_seq=0 out_seq=0 2024-05-11T12:30:51.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.759+0000 7fefc6ffd700 10 client.25235 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.763+0000 7fefc6ffd700 1 -- 172.21.15.47:0/232763183 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5651986802e0 con 0x5651987bc3e0 2024-05-11T12:30:51.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.764+0000 7fefc6ffd700 10 client.25235 handle_command_reply: tid=0 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.764+0000 7feff83e0b80 1 client.25235 shutdown 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 msgr2=0x565198758cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198758cb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fefd8020040 tx=0x7fefd8007970 comp rx=0 tx=0).stop 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 shutdown_connections 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5651987bc3e0 0x5651987be800 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56519873bee0 0x5651987591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5651987260a0 0x565198758770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 --2- 172.21.15.47:0/232763183 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x565198726a60 0x565198758cb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 >> 172.21.15.47:0/232763183 conn(0x565198652e60 msgr2=0x56519873b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7fefc6ffd700 0 client.25235 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:30:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 shutdown_connections 2024-05-11T12:30:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.765+0000 7feff83e0b80 1 -- 172.21.15.47:0/232763183 wait complete. 2024-05-11T12:30:51.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.766+0000 7feff83e0b80 20 client.25235 trim_cache size 0 max 16384 2024-05-11T12:30:51.772 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:30:51.772 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:30:51.772 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:30:51.772 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefd00b2a90 msgr2=0x7fefd00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefd00b2a90 0x7fefd00b4f50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fefe001db20 tx=0x7fefe0009040 comp rx=0 tx=0).stop 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8108470 msgr2=0x7fefe8110e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8108470 0x7fefe8110e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fefe000b070 tx=0x7fefe0004d70 comp rx=0 tx=0).stop 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 shutdown_connections 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fefe8108470 0x7fefe8110e50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fefd00b2a90 0x7fefd00b4f50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fefe806c030 0x7fefe81168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 --2- 172.21.15.47:0/190079988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fefe8105f80 0x7fefe8110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:30:51.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.772+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 >> 172.21.15.47:0/190079988 conn(0x7fefe80793c0 msgr2=0x7fefe8104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:30:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.773+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 shutdown_connections 2024-05-11T12:30:51.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:30:51.773+0000 7fefedb4a700 1 -- 172.21.15.47:0/190079988 wait complete. 2024-05-11T12:30:51.812 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:51 smithi047 ceph-mon[30375]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 762 MiB/s wr, 1.57k op/s 2024-05-11T12:30:51.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:51 smithi064 ceph-mon[34537]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 762 MiB/s wr, 1.57k op/s 2024-05-11T12:30:51.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:51 smithi050 ceph-mon[34072]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 762 MiB/s wr, 1.57k op/s 2024-05-11T12:30:53.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:30:53.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:30:53.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:53 smithi064 ceph-mon[34537]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 825 MiB/s wr, 1.73k op/s 2024-05-11T12:30:53.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:53 smithi050 ceph-mon[34072]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 825 MiB/s wr, 1.73k op/s 2024-05-11T12:30:54.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:53 smithi047 ceph-mon[30375]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 825 MiB/s wr, 1.73k op/s 2024-05-11T12:30:55.047 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:30:55.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:55 smithi047 ceph-mon[30375]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 720 MiB/s wr, 1.63k op/s 2024-05-11T12:30:55.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:55 smithi050 ceph-mon[34072]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 720 MiB/s wr, 1.63k op/s 2024-05-11T12:30:56.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:55 smithi064 ceph-mon[34537]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 720 MiB/s wr, 1.63k op/s 2024-05-11T12:30:57.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:57 smithi050 ceph-mon[34072]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 701 MiB/s wr, 1.66k op/s 2024-05-11T12:30:58.014 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:57 smithi047 ceph-mon[30375]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 701 MiB/s wr, 1.66k op/s 2024-05-11T12:30:58.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:57 smithi064 ceph-mon[34537]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 701 MiB/s wr, 1.66k op/s 2024-05-11T12:30:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:30:59 smithi050 ceph-mon[34072]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 2.26k op/s 2024-05-11T12:31:00.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:30:59 smithi047 ceph-mon[30375]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 2.26k op/s 2024-05-11T12:31:00.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:30:59 smithi064 ceph-mon[34537]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 878 MiB/s wr, 2.26k op/s 2024-05-11T12:31:01.747 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:01 smithi047 ceph-mon[30375]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 702 MiB/s wr, 1.96k op/s 2024-05-11T12:31:01.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.744+0000 7f93b48e0700 1 -- 172.21.15.47:0/5778346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93b0111c40 msgr2=0x7f93b010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:01.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.744+0000 7f93b48e0700 1 --2- 172.21.15.47:0/5778346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93b0111c40 0x7f93b010a550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009990 tx=0x7f93a0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:31:01.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.745+0000 7f93b48e0700 1 -- 172.21.15.47:0/5778346 shutdown_connections 2024-05-11T12:31:01.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.745+0000 7f93b48e0700 1 --2- 172.21.15.47:0/5778346 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b011cd30 0x7f93b011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:01.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.745+0000 7f93b48e0700 1 --2- 172.21.15.47:0/5778346 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93b0111c40 0x7f93b010a550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.745+0000 7f93b48e0700 1 --2- 172.21.15.47:0/5778346 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b01120b0 0x7f93b010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.745+0000 7f93b48e0700 1 -- 172.21.15.47:0/5778346 >> 172.21.15.47:0/5778346 conn(0x7f93b006e530 msgr2=0x7f93b006e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.752+0000 7f93b48e0700 1 -- 172.21.15.47:0/5778346 shutdown_connections 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.752+0000 7f93b48e0700 1 -- 172.21.15.47:0/5778346 wait complete. 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 Processor -- start 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 -- start start 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b0111c40 0x7f93b01ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93b01120b0 0x7f93b01aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b011cd30 0x7f93b01a8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01225a0 con 0x7f93b0111c40 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0122420 con 0x7f93b011cd30 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.754+0000 7f93b48e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0122720 con 0x7f93b01120b0 2024-05-11T12:31:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.755+0000 7f93af59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b011cd30 0x7f93b01a8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.755+0000 7f93af59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b011cd30 0x7f93b01a8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:55766/0 (socket says 172.21.15.47:55766) 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.755+0000 7f93af59e700 1 -- 172.21.15.47:0/409388732 learned_addr learned my addr 172.21.15.47:0/409388732 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.756+0000 7f93af59e700 1 -- 172.21.15.47:0/409388732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93b01120b0 msgr2=0x7f93b01aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.756+0000 7f93af59e700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93b01120b0 0x7f93b01aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.756+0000 7f93af59e700 1 -- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b0111c40 msgr2=0x7f93b01ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.756+0000 7f93af59e700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b0111c40 0x7f93b01ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.756+0000 7f93af59e700 1 -- 172.21.15.47:0/409388732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b01a9020 con 0x7f93b011cd30 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.756+0000 7f93af59e700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b011cd30 0x7f93b01a8840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b9e0 tx=0x7f93a800bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.757+0000 7f9397fff700 1 -- 172.21.15.47:0/409388732 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8019070 con 0x7f93b011cd30 2024-05-11T12:31:01.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.758+0000 7f9397fff700 1 -- 172.21.15.47:0/409388732 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93a8014440 con 0x7f93b011cd30 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.759+0000 7f9397fff700 1 -- 172.21.15.47:0/409388732 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a800ce60 con 0x7f93b011cd30 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.766+0000 7f93b48e0700 1 -- 172.21.15.47:0/409388732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b01a9310 con 0x7f93b011cd30 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.766+0000 7f93b48e0700 1 -- 172.21.15.47:0/409388732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01b5520 con 0x7f93b011cd30 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.772+0000 7f93b48e0700 1 -- 172.21.15.47:0/409388732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b006f970 con 0x7f93b011cd30 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.775+0000 7f9397fff700 1 -- 172.21.15.47:0/409388732 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f93a800c9b0 con 0x7f93b011cd30 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.777+0000 7f9397fff700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f93980b28e0 0x7f93980b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.777+0000 7f9397fff700 1 -- 172.21.15.47:0/409388732 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f93a80a5740 con 0x7f93b011cd30 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.777+0000 7f93aed9d700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f93980b28e0 0x7f93980b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.778+0000 7f93aed9d700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f93980b28e0 0x7f93980b4da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f93a0003800 tx=0x7f93a0009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:01.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:01.782+0000 7f9397fff700 1 -- 172.21.15.47:0/409388732 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93a80a9850 con 0x7f93b011cd30 2024-05-11T12:31:01.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:01 smithi050 ceph-mon[34072]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 702 MiB/s wr, 1.96k op/s 2024-05-11T12:31:02.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:01 smithi064 ceph-mon[34537]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 702 MiB/s wr, 1.96k op/s 2024-05-11T12:31:02.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.402+0000 7f93b48e0700 1 -- 172.21.15.47:0/409388732 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93b01a9d70 con 0x7f93b011cd30 2024-05-11T12:31:02.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.405+0000 7f9397fff700 1 -- 172.21.15.47:0/409388732 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f93a80a9850 con 0x7f93b011cd30 2024-05-11T12:31:02.411 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:31:02.411 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:31:02.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.416+0000 7f9395ffb700 1 -- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f93980b28e0 msgr2=0x7f93980b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.416+0000 7f9395ffb700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f93980b28e0 0x7f93980b4da0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f93a0003800 tx=0x7f93a0009ee0 comp rx=0 tx=0).stop 2024-05-11T12:31:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.416+0000 7f9395ffb700 1 -- 172.21.15.47:0/409388732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b011cd30 msgr2=0x7f93b01a8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:02.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.416+0000 7f9395ffb700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b011cd30 0x7f93b01a8840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b9e0 tx=0x7f93a800bda0 comp rx=0 tx=0).stop 2024-05-11T12:31:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.419+0000 7f9395ffb700 1 -- 172.21.15.47:0/409388732 shutdown_connections 2024-05-11T12:31:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.419+0000 7f9395ffb700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f93980b28e0 0x7f93980b4da0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.419+0000 7f9395ffb700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f93b011cd30 0x7f93b01a8840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.419+0000 7f9395ffb700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93b0111c40 0x7f93b01ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.419+0000 7f9395ffb700 1 --2- 172.21.15.47:0/409388732 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f93b01120b0 0x7f93b01aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.419+0000 7f9395ffb700 1 -- 172.21.15.47:0/409388732 >> 172.21.15.47:0/409388732 conn(0x7f93b006e530 msgr2=0x7f93b010f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:02.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.421+0000 7f9395ffb700 1 -- 172.21.15.47:0/409388732 shutdown_connections 2024-05-11T12:31:02.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:02.421+0000 7f9395ffb700 1 -- 172.21.15.47:0/409388732 wait complete. 2024-05-11T12:31:02.426 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:31:02.680 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:02 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/409388732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:31:02.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:02 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/409388732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:31:03.015 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:02 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/409388732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:31:03.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:03 smithi050 ceph-mon[34072]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 MiB/s wr, 2.30k op/s 2024-05-11T12:31:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:03 smithi047 ceph-mon[30375]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 MiB/s wr, 2.30k op/s 2024-05-11T12:31:04.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:03 smithi064 ceph-mon[34537]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 818 MiB/s wr, 2.30k op/s 2024-05-11T12:31:04.714 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:31:05.908 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:31:05.948 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:05 smithi050 ceph-mon[34072]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 724 MiB/s wr, 2.14k op/s 2024-05-11T12:31:06.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:05 smithi047 ceph-mon[30375]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 724 MiB/s wr, 2.14k op/s 2024-05-11T12:31:06.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:05 smithi064 ceph-mon[34537]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 724 MiB/s wr, 2.14k op/s 2024-05-11T12:31:08.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:07 smithi047 ceph-mon[30375]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 760 MiB/s wr, 2.20k op/s 2024-05-11T12:31:08.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:07 smithi064 ceph-mon[34537]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 760 MiB/s wr, 2.20k op/s 2024-05-11T12:31:08.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:07 smithi050 ceph-mon[34072]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 760 MiB/s wr, 2.20k op/s 2024-05-11T12:31:09.869 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:09 smithi047 ceph-mon[30375]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 853 MiB/s wr, 2.52k op/s 2024-05-11T12:31:10.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:09 smithi064 ceph-mon[34537]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 853 MiB/s wr, 2.52k op/s 2024-05-11T12:31:10.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:09 smithi050 ceph-mon[34072]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 853 MiB/s wr, 2.52k op/s 2024-05-11T12:31:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.923+0000 7f9642b5e700 1 -- 172.21.15.47:0/4036931972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 msgr2=0x7f963c10aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.923+0000 7f9642b5e700 1 --2- 172.21.15.47:0/4036931972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 0x7f963c10aa90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f962c009990 tx=0x7f962c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.924+0000 7f9642b5e700 1 -- 172.21.15.47:0/4036931972 shutdown_connections 2024-05-11T12:31:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.924+0000 7f9642b5e700 1 --2- 172.21.15.47:0/4036931972 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f963c11cd30 0x7f963c11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.924+0000 7f9642b5e700 1 --2- 172.21.15.47:0/4036931972 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f963c111c40 0x7f963c10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.924+0000 7f9642b5e700 1 --2- 172.21.15.47:0/4036931972 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 0x7f963c10aa90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.924+0000 7f9642b5e700 1 -- 172.21.15.47:0/4036931972 >> 172.21.15.47:0/4036931972 conn(0x7f963c06e530 msgr2=0x7f963c06e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.924+0000 7f9642b5e700 1 -- 172.21.15.47:0/4036931972 shutdown_connections 2024-05-11T12:31:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.925+0000 7f9642b5e700 1 -- 172.21.15.47:0/4036931972 wait complete. 2024-05-11T12:31:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.926+0000 7f9642b5e700 1 Processor -- start 2024-05-11T12:31:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.926+0000 7f9642b5e700 1 -- start start 2024-05-11T12:31:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.926+0000 7f9642b5e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f963c111c40 0x7f963c1ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.927+0000 7f9642b5e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 0x7f963c1aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:10.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.927+0000 7f9642b5e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f963c11cd30 0x7f963c1a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:10.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.927+0000 7f9642b5e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f963c122530 con 0x7f963c1120b0 2024-05-11T12:31:10.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.927+0000 7f9642b5e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1223b0 con 0x7f963c11cd30 2024-05-11T12:31:10.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.927+0000 7f9642b5e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f963c1226b0 con 0x7f963c111c40 2024-05-11T12:31:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.932+0000 7f964135b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 0x7f963c1aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.932+0000 7f964135b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 0x7f963c1aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41800/0 (socket says 172.21.15.47:41800) 2024-05-11T12:31:10.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.932+0000 7f964135b700 1 -- 172.21.15.47:0/628795453 learned_addr learned my addr 172.21.15.47:0/628795453 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.935+0000 7f9641b5c700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f963c111c40 0x7f963c1ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:10.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.935+0000 7f964235d700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f963c11cd30 0x7f963c1a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.936+0000 7f9641b5c700 1 -- 172.21.15.47:0/628795453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f963c11cd30 msgr2=0x7f963c1a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.936+0000 7f9641b5c700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f963c11cd30 0x7f963c1a87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.936+0000 7f9641b5c700 1 -- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 msgr2=0x7f963c1aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.936+0000 7f9641b5c700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 0x7f963c1aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.936+0000 7f9641b5c700 1 -- 172.21.15.47:0/628795453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c1a8fd0 con 0x7f963c111c40 2024-05-11T12:31:10.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.939+0000 7f9641b5c700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f963c111c40 0x7f963c1ae670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9f0 tx=0x7f963800edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.946+0000 7f9632ffd700 1 -- 172.21.15.47:0/628795453 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963800cbd0 con 0x7f963c111c40 2024-05-11T12:31:10.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.946+0000 7f9632ffd700 1 -- 172.21.15.47:0/628795453 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f963800cd70 con 0x7f963c111c40 2024-05-11T12:31:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.947+0000 7f9632ffd700 1 -- 172.21.15.47:0/628795453 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638018680 con 0x7f963c111c40 2024-05-11T12:31:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.949+0000 7f9642b5e700 1 -- 172.21.15.47:0/628795453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c1a92c0 con 0x7f963c111c40 2024-05-11T12:31:10.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.949+0000 7f9642b5e700 1 -- 172.21.15.47:0/628795453 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1b54d0 con 0x7f963c111c40 2024-05-11T12:31:10.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.970+0000 7f964e656b80 1 Processor -- start 2024-05-11T12:31:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.974+0000 7f964e656b80 1 -- start start 2024-05-11T12:31:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.974+0000 7f964e656b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5b95570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.974+0000 7f964e656b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8f510 0x560ee5b8f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.974+0000 7f964e656b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b9bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.974+0000 7f964e656b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560ee5ae1940 con 0x560ee5b8f510 2024-05-11T12:31:10.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.974+0000 7f964e656b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560ee5a46da0 con 0x560ee5b95190 2024-05-11T12:31:10.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.974+0000 7f964e656b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560ee5add900 con 0x560ee5b8feb0 2024-05-11T12:31:10.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.977+0000 7f964135b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b9bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:10.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.978+0000 7f9632ffd700 1 -- 172.21.15.47:0/628795453 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f963801f020 con 0x7f963c111c40 2024-05-11T12:31:10.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.979+0000 7f964235d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5b95570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:10.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.979+0000 7f964135b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b9bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:38196/0 (socket says 172.21.15.47:38196) 2024-05-11T12:31:10.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.979+0000 7f964135b700 1 -- 172.21.15.47:0/30639809 learned_addr learned my addr 172.21.15.47:0/30639809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:10.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.979+0000 7f9641b5c700 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8f510 0x560ee5b8f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.983+0000 7f9632ffd700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f96280b28b0 0x7f96280b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:10.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.984+0000 7f9632ffd700 1 -- 172.21.15.47:0/628795453 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f96380e2290 con 0x7f963c111c40 2024-05-11T12:31:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.986+0000 7f964135b700 1 -- 172.21.15.47:0/30639809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 msgr2=0x560ee5b95570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.986+0000 7f964135b700 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5b95570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.986+0000 7f964135b700 1 -- 172.21.15.47:0/30639809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8f510 msgr2=0x560ee5b8f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.986+0000 7f964135b700 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8f510 0x560ee5b8f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.986+0000 7f964135b700 1 -- 172.21.15.47:0/30639809 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee5a2e8d0 con 0x560ee5b8feb0 2024-05-11T12:31:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.988+0000 7f9641b5c700 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8f510 0x560ee5b8f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:31:10.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.989+0000 7f964235d700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f96280b28b0 0x7f96280b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:10.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.991+0000 7f964135b700 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b9bee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f962c0092e0 tx=0x7f962c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a083c031d193aff server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:10.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.991+0000 7f9642b5e700 1 -- 172.21.15.47:0/30639809 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c00bbb0 con 0x560ee5b8feb0 2024-05-11T12:31:10.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.992+0000 7f9642b5e700 1 -- 172.21.15.47:0/30639809 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c00bd50 con 0x560ee5b8feb0 2024-05-11T12:31:10.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.992+0000 7f9642b5e700 1 -- 172.21.15.47:0/30639809 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f962c003e40 con 0x560ee5b8feb0 2024-05-11T12:31:10.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.993+0000 7f964e656b80 1 -- 172.21.15.47:0/30639809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 msgr2=0x560ee5b9bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.993+0000 7f964e656b80 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b9bee0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f962c0092e0 tx=0x7f962c005c00 comp rx=0 tx=0).stop 2024-05-11T12:31:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.994+0000 7f964e656b80 1 -- 172.21.15.47:0/30639809 shutdown_connections 2024-05-11T12:31:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.994+0000 7f964e656b80 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b9bee0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.994+0000 7f964e656b80 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5b95570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.994+0000 7f964e656b80 1 --2- 172.21.15.47:0/30639809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8f510 0x560ee5b8f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:10.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:10.994+0000 7f964e656b80 1 -- 172.21.15.47:0/30639809 >> 172.21.15.47:0/30639809 conn(0x560ee5abbb70 msgr2=0x560ee5b8a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:11.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.001+0000 7f964e656b80 1 -- 172.21.15.47:0/30639809 shutdown_connections 2024-05-11T12:31:11.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.001+0000 7f964235d700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f96280b28b0 0x7f96280b4d70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f963c06caa0 tx=0x7f9634009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.005+0000 7f964e656b80 1 -- 172.21.15.47:0/30639809 wait complete. 2024-05-11T12:31:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.005+0000 7f964e656b80 1 -- start start 2024-05-11T12:31:11.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.005+0000 7f964e656b80 1 Processor -- start 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.006+0000 7f964e656b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b9ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964e656b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b98e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964e656b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b95190 0x560ee5b99370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964e656b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560ee5b9c420 con 0x560ee5b95190 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964e656b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560ee5a9aed0 con 0x560ee5b8f510 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964e656b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560ee5b9c5a0 con 0x560ee5b8feb0 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964e656b80 10 client.0 _ll_register_callbacks cb 0x560ee5a79950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964235d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b95190 0x560ee5b99370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964235d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b95190 0x560ee5b99370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41820/0 (socket says 172.21.15.47:41820) 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.007+0000 7f964235d700 1 -- 172.21.15.47:0/2663370721 learned_addr learned my addr 172.21.15.47:0/2663370721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.008+0000 7f964235d700 1 -- 172.21.15.47:0/2663370721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 msgr2=0x560ee5b98e30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.008+0000 7f964235d700 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b98e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.008+0000 7f964235d700 1 -- 172.21.15.47:0/2663370721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 msgr2=0x560ee5b9ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.008+0000 7f964235d700 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b9ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.008+0000 7f964235d700 1 -- 172.21.15.47:0/2663370721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee5a98760 con 0x560ee5b95190 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.008+0000 7f96257fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:31:11.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.008+0000 7f964235d700 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b95190 0x560ee5b99370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f96340042d0 tx=0x7f9634004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b45eeb56b5ea110 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.014+0000 7f96257fa700 1 -- 172.21.15.47:0/2663370721 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634021070 con 0x560ee5b95190 2024-05-11T12:31:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.014+0000 7f96257fa700 1 -- 172.21.15.47:0/2663370721 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f963401b620 con 0x560ee5b95190 2024-05-11T12:31:11.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.014+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f962c00bd50 con 0x560ee5b95190 2024-05-11T12:31:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.019+0000 7f96257fa700 1 -- 172.21.15.47:0/2663370721 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f963402b050 con 0x560ee5b95190 2024-05-11T12:31:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.019+0000 7f964e656b80 10 client.15663 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:11.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.019+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560ee5bb7ae0 con 0x560ee5b95190 2024-05-11T12:31:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.020+0000 7f96257fa700 1 -- 172.21.15.47:0/2663370721 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f962c00bd50 con 0x560ee5b95190 2024-05-11T12:31:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.021+0000 7f96257fa700 1 -- 172.21.15.47:0/2663370721 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f96340268d0 con 0x560ee5b95190 2024-05-11T12:31:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.021+0000 7f964e656b80 10 client.15663 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.021+0000 7f964e656b80 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:11.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.021+0000 7f964e656b80 20 client.15663 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.021+0000 7f964e656b80 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5bba1c0 0x560ee5bbc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.021+0000 7f964e656b80 4 client.15663 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:31:11.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.021+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ee5aef320 con 0x560ee5bba1c0 2024-05-11T12:31:11.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.026+0000 7f9641b5c700 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5bba1c0 0x560ee5bbc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.041+0000 7f96257fa700 10 client.15663 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:11.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.041+0000 7f9641b5c700 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5bba1c0 0x560ee5bbc650 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0f564341948aee server_cookie=cae725bbef237cf6 in_seq=0 out_seq=0 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.044+0000 7f96257fa700 1 -- 172.21.15.47:0/2663370721 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560ee5aef320 con 0x560ee5bba1c0 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.044+0000 7f96257fa700 10 client.15663 handle_command_reply: tid=0 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 client.15663 shutdown 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b95190 msgr2=0x560ee5b99370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b95190 0x560ee5b99370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f96340042d0 tx=0x7f9634004ad0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 shutdown_connections 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5bba1c0 0x560ee5bbc650 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b95190 0x560ee5b99370 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b9ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 --2- 172.21.15.47:0/2663370721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b8feb0 0x560ee5b98e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.045+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 >> 172.21.15.47:0/2663370721 conn(0x560ee5abbb70 msgr2=0x560ee5b8e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:11.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.046+0000 7f96257fa700 0 client.15663 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.050+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 shutdown_connections 2024-05-11T12:31:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.050+0000 7f964e656b80 1 -- 172.21.15.47:0/2663370721 wait complete. 2024-05-11T12:31:11.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.050+0000 7f964e656b80 20 client.15663 trim_cache size 0 max 16384 2024-05-11T12:31:11.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.089+0000 7f964e656b80 1 Processor -- start 2024-05-11T12:31:11.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.090+0000 7f964e656b80 1 -- start start 2024-05-11T12:31:11.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.090+0000 7f964e656b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b8a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.090+0000 7f964e656b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5bc1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.091+0000 7f964e656b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b95190 0x560ee5bc2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.091+0000 7f9641b5c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b8a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.091+0000 7f9641b5c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b8a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:47288/0 (socket says 172.21.15.47:47288) 2024-05-11T12:31:11.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.091+0000 7f9641b5c700 1 -- 172.21.15.47:0/2598390212 learned_addr learned my addr 172.21.15.47:0/2598390212 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:11.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.091+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f963c117990 con 0x560ee5b8feb0 2024-05-11T12:31:11.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.091+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f963c11a250 con 0x560ee5b8f510 2024-05-11T12:31:11.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.092+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f963c118040 con 0x560ee5b95190 2024-05-11T12:31:11.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.094+0000 7f964135b700 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5bc1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.097+0000 7f964235d700 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b95190 0x560ee5bc2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.097+0000 7f9641b5c700 1 -- 172.21.15.47:0/2598390212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b95190 msgr2=0x560ee5bc2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.097+0000 7f9641b5c700 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b95190 0x560ee5bc2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.097+0000 7f9641b5c700 1 -- 172.21.15.47:0/2598390212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 msgr2=0x560ee5bc1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.097+0000 7f9641b5c700 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5bc1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.097+0000 7f9641b5c700 1 -- 172.21.15.47:0/2598390212 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ee5bbccd0 con 0x560ee5b8f510 2024-05-11T12:31:11.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.105+0000 7f9641b5c700 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b8a5c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f963800c5b0 tx=0x7f96380cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d477a126a2216b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:11.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.108+0000 7f96277fe700 1 -- 172.21.15.47:0/2598390212 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380cc0b0 con 0x560ee5b8f510 2024-05-11T12:31:11.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.108+0000 7f96277fe700 1 -- 172.21.15.47:0/2598390212 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ee5aef320 con 0x560ee5b8f510 2024-05-11T12:31:11.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.109+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 msgr2=0x560ee5b8a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:11.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.109+0000 7f964e656b80 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b8a5c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f963800c5b0 tx=0x7f96380cc040 comp rx=0 tx=0).stop 2024-05-11T12:31:11.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.109+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 shutdown_connections 2024-05-11T12:31:11.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.109+0000 7f964e656b80 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5b95190 0x560ee5bc2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.109+0000 7f964e656b80 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b8f510 0x560ee5b8a5c0 secure :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f963800c5b0 tx=0x7f96380cc040 comp rx=0 tx=0).stop 2024-05-11T12:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.109+0000 7f964e656b80 1 --2- 172.21.15.47:0/2598390212 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5bc1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.109+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 >> 172.21.15.47:0/2598390212 conn(0x560ee5abbb70 msgr2=0x560ee5b89eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:11.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.113+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 shutdown_connections 2024-05-11T12:31:11.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.117+0000 7f964e656b80 1 -- 172.21.15.47:0/2598390212 wait complete. 2024-05-11T12:31:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.117+0000 7f964e656b80 1 -- start start 2024-05-11T12:31:11.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.118+0000 7f964e656b80 1 Processor -- start 2024-05-11T12:31:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.121+0000 7f964e656b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5ba1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.122+0000 7f964e656b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5bc14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.122+0000 7f964e656b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5bc1b90 0x560ee5c23ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.122+0000 7f964e656b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f963c118710 con 0x560ee5b8feb0 2024-05-11T12:31:11.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.122+0000 7f964e656b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f963c119b80 con 0x560ee5b95190 2024-05-11T12:31:11.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.122+0000 7f964e656b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f963c118de0 con 0x560ee5bc1b90 2024-05-11T12:31:11.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.122+0000 7f964e656b80 10 client.0 _ll_register_callbacks cb 0x560ee5a79950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:11.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.123+0000 7f9641b5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5ba1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.123+0000 7f9641b5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5ba1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41840/0 (socket says 172.21.15.47:41840) 2024-05-11T12:31:11.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.123+0000 7f9641b5c700 1 -- 172.21.15.47:0/1654909893 learned_addr learned my addr 172.21.15.47:0/1654909893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:11.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.123+0000 7f964135b700 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5bc14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.124+0000 7f964235d700 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5bc1b90 0x560ee5c23ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.124+0000 7f9641b5c700 1 -- 172.21.15.47:0/1654909893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5bc1b90 msgr2=0x560ee5c23ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:11.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.124+0000 7f9641b5c700 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5bc1b90 0x560ee5c23ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.124+0000 7f9641b5c700 1 -- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 msgr2=0x560ee5bc14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:11.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.124+0000 7f9641b5c700 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5bc14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.124+0000 7f9641b5c700 1 -- 172.21.15.47:0/1654909893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ee5bc22d0 con 0x560ee5b8feb0 2024-05-11T12:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.125+0000 7f9641b5c700 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5ba1ed0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f96380a9710 tx=0x7f9638091340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69339b25d150bd7e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.125+0000 7f96257fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.126+0000 7f96257fa700 1 -- 172.21.15.47:0/1654909893 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96380a9780 con 0x560ee5b8feb0 2024-05-11T12:31:11.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.126+0000 7f96257fa700 1 -- 172.21.15.47:0/1654909893 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f96380a8bf0 con 0x560ee5b8feb0 2024-05-11T12:31:11.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.129+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ee5ae93c0 con 0x560ee5b8feb0 2024-05-11T12:31:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.133+0000 7f96257fa700 1 -- 172.21.15.47:0/1654909893 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f96380a9e80 con 0x560ee5b8feb0 2024-05-11T12:31:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.134+0000 7f964e656b80 10 client.15669 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:11.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.134+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560ee5c25110 con 0x560ee5b8feb0 2024-05-11T12:31:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.136+0000 7f96257fa700 1 -- 172.21.15.47:0/1654909893 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560ee5ae93c0 con 0x560ee5b8feb0 2024-05-11T12:31:11.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.137+0000 7f96257fa700 1 -- 172.21.15.47:0/1654909893 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f96380aaab0 con 0x560ee5b8feb0 2024-05-11T12:31:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.138+0000 7f964e656b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.138+0000 7f964e656b80 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.138+0000 7f964e656b80 20 client.15669 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.138+0000 7f964e656b80 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5c25650 0x560ee5c27a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.138+0000 7f964e656b80 4 client.15669 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:31:11.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.138+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ee5961250 con 0x560ee5c25650 2024-05-11T12:31:11.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.140+0000 7f964135b700 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5c25650 0x560ee5c27a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.149+0000 7f964135b700 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5c25650 0x560ee5c27a70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bccccec461052127 server_cookie=8194315d4a1b2c57 in_seq=0 out_seq=0 2024-05-11T12:31:11.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.149+0000 7f96257fa700 10 client.15669 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.153+0000 7f96257fa700 1 -- 172.21.15.47:0/1654909893 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ee5961250 con 0x560ee5c25650 2024-05-11T12:31:11.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.153+0000 7f96257fa700 10 client.15669 handle_command_reply: tid=0 2024-05-11T12:31:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.153+0000 7f964e656b80 1 client.15669 shutdown 2024-05-11T12:31:11.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 msgr2=0x560ee5ba1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5ba1ed0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f96380a9710 tx=0x7f9638091340 comp rx=0 tx=0).stop 2024-05-11T12:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 shutdown_connections 2024-05-11T12:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560ee5c25650 0x560ee5c27a70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560ee5bc1b90 0x560ee5c23ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560ee5b8feb0 0x560ee5ba1ed0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 --2- 172.21.15.47:0/1654909893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560ee5b95190 0x560ee5bc14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f96257fa700 0 client.15669 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 >> 172.21.15.47:0/1654909893 conn(0x560ee5abbb70 msgr2=0x560ee5b8e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 shutdown_connections 2024-05-11T12:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.154+0000 7f964e656b80 1 -- 172.21.15.47:0/1654909893 wait complete. 2024-05-11T12:31:11.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.155+0000 7f964e656b80 20 client.15669 trim_cache size 0 max 16384 2024-05-11T12:31:11.158 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 -- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f96280b28b0 msgr2=0x7f96280b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f96280b28b0 0x7f96280b4d70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f963c06caa0 tx=0x7f9634009040 comp rx=0 tx=0).stop 2024-05-11T12:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 -- 172.21.15.47:0/628795453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f963c111c40 msgr2=0x7f963c1ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f963c111c40 0x7f963c1ae670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9f0 tx=0x7f963800edb0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 -- 172.21.15.47:0/628795453 shutdown_connections 2024-05-11T12:31:11.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f96280b28b0 0x7f96280b4d70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f963c11cd30 0x7f963c1a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f963c111c40 0x7f963c1ae670 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 --2- 172.21.15.47:0/628795453 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f963c1120b0 0x7f963c1aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 -- 172.21.15.47:0/628795453 >> 172.21.15.47:0/628795453 conn(0x7f963c06e530 msgr2=0x7f963c10f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 -- 172.21.15.47:0/628795453 shutdown_connections 2024-05-11T12:31:11.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:11.157+0000 7f9630ff9700 1 -- 172.21.15.47:0/628795453 wait complete. 2024-05-11T12:31:12.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:11 smithi047 ceph-mon[30375]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 712 MiB/s wr, 2.05k op/s 2024-05-11T12:31:12.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:11 smithi064 ceph-mon[34537]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 712 MiB/s wr, 2.05k op/s 2024-05-11T12:31:12.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:11 smithi050 ceph-mon[34072]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 712 MiB/s wr, 2.05k op/s 2024-05-11T12:31:13.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:12 smithi064 ceph-mon[34537]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 875 MiB/s wr, 2.40k op/s 2024-05-11T12:31:13.198 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:12 smithi050 ceph-mon[34072]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 875 MiB/s wr, 2.40k op/s 2024-05-11T12:31:13.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:12 smithi047 ceph-mon[30375]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 875 MiB/s wr, 2.40k op/s 2024-05-11T12:31:14.491 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:31:14.492 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:31:15.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:15 smithi050 ceph-mon[34072]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 744 MiB/s wr, 1.95k op/s 2024-05-11T12:31:15.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:15 smithi047 ceph-mon[30375]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 744 MiB/s wr, 1.95k op/s 2024-05-11T12:31:15.554 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:31:15.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:15 smithi064 ceph-mon[34537]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 744 MiB/s wr, 1.95k op/s 2024-05-11T12:31:17.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:17 smithi047 ceph-mon[30375]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.92k op/s 2024-05-11T12:31:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:17 smithi050 ceph-mon[34072]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.92k op/s 2024-05-11T12:31:17.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:17 smithi064 ceph-mon[34537]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.92k op/s 2024-05-11T12:31:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:19 smithi050 ceph-mon[34072]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 865 MiB/s wr, 2.02k op/s 2024-05-11T12:31:19.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:19 smithi047 ceph-mon[30375]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 865 MiB/s wr, 2.02k op/s 2024-05-11T12:31:19.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:19 smithi064 ceph-mon[34537]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 865 MiB/s wr, 2.02k op/s 2024-05-11T12:31:21.298 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:21 smithi047 ceph-mon[30375]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 777 MiB/s wr, 1.63k op/s 2024-05-11T12:31:21.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:21 smithi050 ceph-mon[34072]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 777 MiB/s wr, 1.63k op/s 2024-05-11T12:31:21.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:21 smithi064 ceph-mon[34537]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 777 MiB/s wr, 1.63k op/s 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.396+0000 7f26fb14e700 1 -- 172.21.15.47:0/1013817816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f4111d30 msgr2=0x7f26f410a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.396+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1013817816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f4111d30 0x7f26f410a550 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f26e4009990 tx=0x7f26e4009ca0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.398+0000 7f26fb14e700 1 -- 172.21.15.47:0/1013817816 shutdown_connections 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.398+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1013817816 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 0x7f26f411f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.398+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1013817816 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f4111d30 0x7f26f410a550 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.398+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1013817816 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26f410aa90 0x7f26f410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.398+0000 7f26fb14e700 1 -- 172.21.15.47:0/1013817816 >> 172.21.15.47:0/1013817816 conn(0x7f26f406e6c0 msgr2=0x7f26f406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.398+0000 7f26fb14e700 1 -- 172.21.15.47:0/1013817816 shutdown_connections 2024-05-11T12:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.398+0000 7f26fb14e700 1 -- 172.21.15.47:0/1013817816 wait complete. 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.403+0000 7f26fb14e700 1 Processor -- start 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.403+0000 7f26fb14e700 1 -- start start 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.403+0000 7f26fb14e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26f410aa90 0x7f26f41ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.403+0000 7f26fb14e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f4111d30 0x7f26f41aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.403+0000 7f26fb14e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 0x7f26f41a8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.403+0000 7f26fb14e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4122610 con 0x7f26f411cd30 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.404+0000 7f26fb14e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4122490 con 0x7f26f410aa90 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.404+0000 7f26fb14e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26f4122790 con 0x7f26f4111d30 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.404+0000 7f26f96eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 0x7f26f41a8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.404+0000 7f26f96eb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 0x7f26f41a8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41586/0 (socket says 172.21.15.47:41586) 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.404+0000 7f26f96eb700 1 -- 172.21.15.47:0/1991298034 learned_addr learned my addr 172.21.15.47:0/1991298034 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.405+0000 7f26f96eb700 1 -- 172.21.15.47:0/1991298034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f4111d30 msgr2=0x7f26f41aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.405+0000 7f26f96eb700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f4111d30 0x7f26f41aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.405+0000 7f26f96eb700 1 -- 172.21.15.47:0/1991298034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26f410aa90 msgr2=0x7f26f41ae750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:31:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.405+0000 7f26f96eb700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26f410aa90 0x7f26f41ae750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.405+0000 7f26f96eb700 1 -- 172.21.15.47:0/1991298034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f41a9070 con 0x7f26f411cd30 2024-05-11T12:31:22.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.425+0000 7f26f96eb700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 0x7f26f41a8910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b9e0 tx=0x7f26ec00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:22.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.429+0000 7f26f1ffb700 1 -- 172.21.15.47:0/1991298034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec019070 con 0x7f26f411cd30 2024-05-11T12:31:22.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.429+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f41a9360 con 0x7f26f411cd30 2024-05-11T12:31:22.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.429+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f4112510 con 0x7f26f411cd30 2024-05-11T12:31:22.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.430+0000 7f26f1ffb700 1 -- 172.21.15.47:0/1991298034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26ec014440 con 0x7f26f411cd30 2024-05-11T12:31:22.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.431+0000 7f26f1ffb700 1 -- 172.21.15.47:0/1991298034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec00ce60 con 0x7f26f411cd30 2024-05-11T12:31:22.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.457+0000 7f26f1ffb700 1 -- 172.21.15.47:0/1991298034 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f26ec00c9b0 con 0x7f26f411cd30 2024-05-11T12:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.477+0000 7f27059e4b80 1 Processor -- start 2024-05-11T12:31:22.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.478+0000 7f27059e4b80 1 -- start start 2024-05-11T12:31:22.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.479+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 0x5617dd79af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.479+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795120 0x5617dd7955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.479+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.479+0000 7f27059e4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5617dd6e78e0 con 0x5617dd79ab50 2024-05-11T12:31:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.479+0000 7f27059e4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5617dd686840 con 0x5617dd795120 2024-05-11T12:31:22.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.479+0000 7f27059e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5617dd6e38a0 con 0x5617dd795ae0 2024-05-11T12:31:22.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.480+0000 7f26f1ffb700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26dc0b27d0 0x7f26dc0b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.480+0000 7f26f1ffb700 1 -- 172.21.15.47:0/1991298034 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f26ec0e1a30 con 0x7f26f411cd30 2024-05-11T12:31:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.482+0000 7f26f96eb700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26dc0b27d0 0x7f26dc0b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.482+0000 7f26f8eea700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795120 0x5617dd7955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.482+0000 7f26f8eea700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795120 0x5617dd7955a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:38284/0 (socket says 172.21.15.47:38284) 2024-05-11T12:31:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.482+0000 7f26f8eea700 1 -- 172.21.15.47:0/3731444156 learned_addr learned my addr 172.21.15.47:0/3731444156 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:22.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.482+0000 7f26f3fff700 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.486+0000 7f26f8eea700 1 -- 172.21.15.47:0/3731444156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 msgr2=0x5617dd7a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.486+0000 7f26f8eea700 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.486+0000 7f26f8eea700 1 -- 172.21.15.47:0/3731444156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 msgr2=0x5617dd79af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.486+0000 7f26f8eea700 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 0x5617dd79af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.487+0000 7f26f8eea700 1 -- 172.21.15.47:0/3731444156 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dd634950 con 0x5617dd795120 2024-05-11T12:31:22.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.488+0000 7f26f8eea700 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795120 0x5617dd7955a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26e40059d0 tx=0x7f26e40096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b46cf40de86ebe4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:22.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.490+0000 7f26f96eb700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26dc0b27d0 0x7f26dc0b4c90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0a8c80 tx=0x7f26ec004a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.490+0000 7f26daffd700 1 -- 172.21.15.47:0/3731444156 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4026070 con 0x5617dd795120 2024-05-11T12:31:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.491+0000 7f26daffd700 1 -- 172.21.15.47:0/3731444156 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e400bd70 con 0x5617dd795120 2024-05-11T12:31:22.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.491+0000 7f26daffd700 1 -- 172.21.15.47:0/3731444156 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e4004230 con 0x5617dd795120 2024-05-11T12:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.495+0000 7f27059e4b80 1 -- 172.21.15.47:0/3731444156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795120 msgr2=0x5617dd7955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.495+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795120 0x5617dd7955a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26e40059d0 tx=0x7f26e40096a0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.495+0000 7f27059e4b80 1 -- 172.21.15.47:0/3731444156 shutdown_connections 2024-05-11T12:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.495+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.495+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 0x5617dd79af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.495+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3731444156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795120 0x5617dd7955a0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f26e40059d0 tx=0x7f26e40096a0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.495+0000 7f27059e4b80 1 -- 172.21.15.47:0/3731444156 >> 172.21.15.47:0/3731444156 conn(0x5617dd6c1ee0 msgr2=0x5617dd78fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:22.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.500+0000 7f27059e4b80 1 -- 172.21.15.47:0/3731444156 shutdown_connections 2024-05-11T12:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.501+0000 7f27059e4b80 1 -- 172.21.15.47:0/3731444156 wait complete. 2024-05-11T12:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.502+0000 7f27059e4b80 1 -- start start 2024-05-11T12:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.502+0000 7f27059e4b80 1 Processor -- start 2024-05-11T12:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.503+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.503+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd79ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.503+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79f010 0x5617dd79f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.503+0000 7f27059e4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5617dd7a1e50 con 0x5617dd79f010 2024-05-11T12:31:22.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.508+0000 7f26f8eea700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd79ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.509+0000 7f26f8eea700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd79ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:38300/0 (socket says 172.21.15.47:38300) 2024-05-11T12:31:22.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.509+0000 7f26f8eea700 1 -- 172.21.15.47:0/3703834934 learned_addr learned my addr 172.21.15.47:0/3703834934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:22.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.509+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5617dd64dcb0 con 0x5617dd79ab50 2024-05-11T12:31:22.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.509+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5617dd7a1fd0 con 0x5617dd795ae0 2024-05-11T12:31:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.513+0000 7f27059e4b80 10 client.0 _ll_register_callbacks cb 0x5617dd67f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:22.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.510+0000 7f26f3fff700 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79f010 0x5617dd79f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.513+0000 7f26f3fff700 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.517+0000 7f26f3fff700 1 -- 172.21.15.47:0/3703834934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 msgr2=0x5617dd7a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.517+0000 7f26f3fff700 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.517+0000 7f26f3fff700 1 -- 172.21.15.47:0/3703834934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 msgr2=0x5617dd79ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.517+0000 7f26f3fff700 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd79ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.518+0000 7f26f3fff700 1 -- 172.21.15.47:0/3703834934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dd683710 con 0x5617dd79f010 2024-05-11T12:31:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.525+0000 7f26f3fff700 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79f010 0x5617dd79f490 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26e8005950 tx=0x7f26e80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd95c4ca71025849 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.525+0000 7f26d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:31:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.526+0000 7f26d3fff700 1 -- 172.21.15.47:0/3703834934 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8004050 con 0x5617dd79f010 2024-05-11T12:31:22.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.526+0000 7f26d3fff700 1 -- 172.21.15.47:0/3703834934 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f26e80041f0 con 0x5617dd79f010 2024-05-11T12:31:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.527+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e400bd70 con 0x5617dd79f010 2024-05-11T12:31:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.528+0000 7f26d3fff700 1 -- 172.21.15.47:0/3703834934 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f26e8024050 con 0x5617dd79f010 2024-05-11T12:31:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.528+0000 7f27059e4b80 10 client.15675 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.529+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5617dd7acd20 con 0x5617dd79f010 2024-05-11T12:31:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f26d3fff700 1 -- 172.21.15.47:0/3703834934 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e400bd70 con 0x5617dd79f010 2024-05-11T12:31:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f26d3fff700 1 -- 172.21.15.47:0/3703834934 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f26e8003680 con 0x5617dd79f010 2024-05-11T12:31:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f27059e4b80 10 client.15675 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f27059e4b80 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f27059e4b80 20 client.15675 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd7af360 0x5617dd7b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f27059e4b80 4 client.15675 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:31:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.534+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617dd6f52c0 con 0x5617dd7af360 2024-05-11T12:31:22.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.544+0000 7f26f8eea700 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd7af360 0x5617dd7b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.557+0000 7f26f8eea700 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd7af360 0x5617dd7b1780 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273e31aaab654656 server_cookie=c5af46a8d5d60178 in_seq=0 out_seq=0 2024-05-11T12:31:22.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.557+0000 7f26d3fff700 10 client.15675 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.561+0000 7f26d3fff700 1 -- 172.21.15.47:0/3703834934 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617dd6f52c0 con 0x5617dd7af360 2024-05-11T12:31:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.561+0000 7f26d3fff700 10 client.15675 handle_command_reply: tid=0 2024-05-11T12:31:22.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.562+0000 7f27059e4b80 1 client.15675 shutdown 2024-05-11T12:31:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.562+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79f010 msgr2=0x5617dd79f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.562+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79f010 0x5617dd79f490 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26e8005950 tx=0x7f26e80096c0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.569+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 shutdown_connections 2024-05-11T12:31:22.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.569+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd7af360 0x5617dd7b1780 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.569+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79f010 0x5617dd79f490 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.569+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd795ae0 0x5617dd7a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.569+0000 7f27059e4b80 1 --2- 172.21.15.47:0/3703834934 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd79ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.569+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 >> 172.21.15.47:0/3703834934 conn(0x5617dd6c1ee0 msgr2=0x5617dd78a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:22.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.570+0000 7f26d3fff700 0 client.15675 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:22.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.571+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 shutdown_connections 2024-05-11T12:31:22.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.571+0000 7f27059e4b80 1 -- 172.21.15.47:0/3703834934 wait complete. 2024-05-11T12:31:22.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.572+0000 7f27059e4b80 20 client.15675 trim_cache size 0 max 16384 2024-05-11T12:31:22.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.629+0000 7f27059e4b80 1 Processor -- start 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.629+0000 7f27059e4b80 1 -- start start 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd795ae0 0x5617dd79e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd7a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f27059e4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000b7f0 con 0x5617dd795ae0 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f27059e4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000e070 con 0x5617dd79ab50 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f27059e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000be60 con 0x5617dd79f010 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f26f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f26f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:58860/0 (socket says 172.21.15.47:58860) 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f26f3fff700 1 -- 172.21.15.47:0/752974524 learned_addr learned my addr 172.21.15.47:0/752974524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.630+0000 7f26f3fff700 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd795ae0 0x5617dd79e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.631+0000 7f26f3fff700 1 -- 172.21.15.47:0/752974524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 msgr2=0x5617dd7a09c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.631+0000 7f26f3fff700 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd7a09c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.631+0000 7f26f3fff700 1 -- 172.21.15.47:0/752974524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd795ae0 msgr2=0x5617dd79e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.631+0000 7f26f3fff700 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd795ae0 0x5617dd79e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.631+0000 7f26f3fff700 1 -- 172.21.15.47:0/752974524 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617dd78c610 con 0x5617dd79f010 2024-05-11T12:31:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.631+0000 7f26f3fff700 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7b70c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001eb0 tx=0x7f26e8014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b013a00af0a071d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:22.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.632+0000 7f26da7fc700 1 -- 172.21.15.47:0/752974524 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8017030 con 0x5617dd79f010 2024-05-11T12:31:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.633+0000 7f26da7fc700 1 -- 172.21.15.47:0/752974524 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e8009260 con 0x5617dd79f010 2024-05-11T12:31:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.633+0000 7f26da7fc700 1 -- 172.21.15.47:0/752974524 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e800fdc0 con 0x5617dd79f010 2024-05-11T12:31:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.634+0000 7f27059e4b80 1 -- 172.21.15.47:0/752974524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 msgr2=0x5617dd7b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.634+0000 7f27059e4b80 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7b70c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f26e8001eb0 tx=0x7f26e8014040 comp rx=0 tx=0).stop 2024-05-11T12:31:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.635+0000 7f27059e4b80 1 -- 172.21.15.47:0/752974524 shutdown_connections 2024-05-11T12:31:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.635+0000 7f27059e4b80 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7b70c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.635+0000 7f27059e4b80 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd795ae0 0x5617dd79e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.635+0000 7f27059e4b80 1 --2- 172.21.15.47:0/752974524 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd79ab50 0x5617dd7a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.635+0000 7f27059e4b80 1 -- 172.21.15.47:0/752974524 >> 172.21.15.47:0/752974524 conn(0x5617dd6c1ee0 msgr2=0x5617dd7a02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.636+0000 7f27059e4b80 1 -- 172.21.15.47:0/752974524 shutdown_connections 2024-05-11T12:31:22.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.636+0000 7f27059e4b80 1 -- 172.21.15.47:0/752974524 wait complete. 2024-05-11T12:31:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.637+0000 7f27059e4b80 1 -- start start 2024-05-11T12:31:22.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.637+0000 7f27059e4b80 1 Processor -- start 2024-05-11T12:31:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.638+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795ae0 0x5617dd7a7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.638+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 0x5617dd7a2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.638+0000 7f27059e4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.638+0000 7f27059e4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000c530 con 0x5617dd79ab50 2024-05-11T12:31:22.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.638+0000 7f27059e4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000d9a0 con 0x5617dd795ae0 2024-05-11T12:31:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.638+0000 7f27059e4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000cc00 con 0x5617dd79f010 2024-05-11T12:31:22.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.638+0000 7f27059e4b80 10 client.0 _ll_register_callbacks cb 0x5617dd67f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7a29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:58870/0 (socket says 172.21.15.47:58870) 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 -- 172.21.15.47:0/72252459 learned_addr learned my addr 172.21.15.47:0/72252459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795ae0 0x5617dd7a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 -- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795ae0 msgr2=0x5617dd7a7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795ae0 0x5617dd7a7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 -- 172.21.15.47:0/72252459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 msgr2=0x5617dd7a2280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 0x5617dd7a2280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.639+0000 7f26f3fff700 1 -- 172.21.15.47:0/72252459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617dd79fa90 con 0x5617dd79f010 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.640+0000 7f26f3fff700 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7a29c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26e800fac0 tx=0x7f26e800f960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce1a07bbe57ed43e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.641+0000 7f26d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:31:22.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.641+0000 7f26d3fff700 1 -- 172.21.15.47:0/72252459 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e802f040 con 0x5617dd79f010 2024-05-11T12:31:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.641+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5617dd6ef360 con 0x5617dd79f010 2024-05-11T12:31:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.641+0000 7f26d3fff700 1 -- 172.21.15.47:0/72252459 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f26e8005040 con 0x5617dd79f010 2024-05-11T12:31:22.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.641+0000 7f26d3fff700 1 -- 172.21.15.47:0/72252459 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f26e8005240 con 0x5617dd79f010 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.643+0000 7f27059e4b80 10 client.25312 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.643+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5617dd81ac40 con 0x5617dd79f010 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.643+0000 7f26d3fff700 1 -- 172.21.15.47:0/72252459 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5617dd6ef360 con 0x5617dd79f010 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.645+0000 7f26d3fff700 1 -- 172.21.15.47:0/72252459 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f26e8004000 con 0x5617dd79f010 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.645+0000 7f27059e4b80 10 client.25312 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.645+0000 7f27059e4b80 10 client.25312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.645+0000 7f27059e4b80 20 client.25312 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:22.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.645+0000 7f27059e4b80 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd81b1c0 0x5617dd81d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:22.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.645+0000 7f27059e4b80 4 client.25312 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:31:22.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.645+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5617dd5672d0 con 0x5617dd81b1c0 2024-05-11T12:31:22.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.646+0000 7f26f8eea700 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd81b1c0 0x5617dd81d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:22.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.648+0000 7f26f8eea700 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd81b1c0 0x5617dd81d5e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c56c987da379a81 server_cookie=40751b9c0efc17cf in_seq=0 out_seq=0 2024-05-11T12:31:22.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.649+0000 7f26d3fff700 10 client.25312 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:22.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.953+0000 7f26d3fff700 1 -- 172.21.15.47:0/72252459 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5617dd5672d0 con 0x5617dd81b1c0 2024-05-11T12:31:22.956 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:31:22.957 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:31:22.957 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "c4efed2d-90f5-4bcc-be70-06d8b3b3c412", 2024-05-11T12:31:22.957 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:31:22.957 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:31:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f26d3fff700 10 client.25312 handle_command_reply: tid=0 2024-05-11T12:31:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 client.25312 shutdown 2024-05-11T12:31:22.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 msgr2=0x5617dd7a29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7a29c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26e800fac0 tx=0x7f26e800f960 comp rx=0 tx=0).stop 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 shutdown_connections 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5617dd81b1c0 0x5617dd81d5e0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5617dd79f010 0x5617dd7a29c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5617dd795ae0 0x5617dd7a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 --2- 172.21.15.47:0/72252459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5617dd79ab50 0x5617dd7a2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.954+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 >> 172.21.15.47:0/72252459 conn(0x5617dd6c1ee0 msgr2=0x5617dd7949a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.955+0000 7f26d3fff700 0 client.25312 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.955+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 shutdown_connections 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.955+0000 7f27059e4b80 1 -- 172.21.15.47:0/72252459 wait complete. 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.955+0000 7f27059e4b80 20 client.25312 trim_cache size 0 max 16384 2024-05-11T12:31:22.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26dc0b27d0 msgr2=0x7f26dc0b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26dc0b27d0 0x7f26dc0b4c90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0a8c80 tx=0x7f26ec004a90 comp rx=0 tx=0).stop 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 msgr2=0x7f26f41a8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 0x7f26f41a8910 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00b9e0 tx=0x7f26ec00bda0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 shutdown_connections 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f26f411cd30 0x7f26f41a8910 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f26dc0b27d0 0x7f26dc0b4c90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f26f410aa90 0x7f26f41ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 --2- 172.21.15.47:0/1991298034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f26f4111d30 0x7f26f41aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:22.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.956+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 >> 172.21.15.47:0/1991298034 conn(0x7f26f406e6c0 msgr2=0x7f26f410f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:22.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.957+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 shutdown_connections 2024-05-11T12:31:22.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:22.960+0000 7f26fb14e700 1 -- 172.21.15.47:0/1991298034 wait complete. 2024-05-11T12:31:23.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:23 smithi047 ceph-mon[30375]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 901 MiB/s wr, 1.68k op/s 2024-05-11T12:31:23.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:23 smithi064 ceph-mon[34537]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 901 MiB/s wr, 1.68k op/s 2024-05-11T12:31:23.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:23 smithi050 ceph-mon[34072]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 901 MiB/s wr, 1.68k op/s 2024-05-11T12:31:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:24 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:31:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:24 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:31:24.531 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:24 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:31:24.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:24 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:31:24.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:24 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:31:24.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:24 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:31:24.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:24 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:31:24.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:24 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:31:24.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:24 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:31:25.533 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:25 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:31:25.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:25 smithi047 ceph-mon[30375]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 755 MiB/s wr, 1.26k op/s 2024-05-11T12:31:25.599 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:25 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:31:25.599 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:25 smithi050 ceph-mon[34072]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 755 MiB/s wr, 1.26k op/s 2024-05-11T12:31:25.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:25 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:31:25.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:25 smithi064 ceph-mon[34537]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 755 MiB/s wr, 1.26k op/s 2024-05-11T12:31:25.668 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:31:26.741 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:31:27.567 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:27 smithi047 ceph-mon[30375]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 767 MiB/s wr, 1.31k op/s 2024-05-11T12:31:27.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:27 smithi064 ceph-mon[34537]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 767 MiB/s wr, 1.31k op/s 2024-05-11T12:31:27.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:27 smithi050 ceph-mon[34072]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 767 MiB/s wr, 1.31k op/s 2024-05-11T12:31:29.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:29 smithi047 ceph-mon[30375]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 846 MiB/s wr, 1.64k op/s 2024-05-11T12:31:29.534 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:29 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:31:29.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:29 smithi064 ceph-mon[34537]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 846 MiB/s wr, 1.64k op/s 2024-05-11T12:31:29.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:29 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:31:29.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:29 smithi050 ceph-mon[34072]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 846 MiB/s wr, 1.64k op/s 2024-05-11T12:31:29.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:29 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:31:31.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:31 smithi064 ceph-mon[34537]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 1.51k op/s 2024-05-11T12:31:31.629 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:31 smithi047 ceph-mon[30375]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 1.51k op/s 2024-05-11T12:31:31.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:31 smithi050 ceph-mon[34072]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 760 MiB/s wr, 1.51k op/s 2024-05-11T12:31:31.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 -- 172.21.15.47:0/2919260018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c10a7b0 msgr2=0x7ff18c10ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:31.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 --2- 172.21.15.47:0/2919260018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c10a7b0 0x7ff18c10ab90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff17c008580 tx=0x7ff17c00ac60 comp rx=0 tx=0).stop 2024-05-11T12:31:31.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 -- 172.21.15.47:0/2919260018 shutdown_connections 2024-05-11T12:31:31.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 --2- 172.21.15.47:0/2919260018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c10ef10 0x7ff18c07b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:31.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 --2- 172.21.15.47:0/2919260018 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c10a7b0 0x7ff18c10ab90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:31.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 --2- 172.21.15.47:0/2919260018 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 0x7ff18c10e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:31.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 -- 172.21.15.47:0/2919260018 >> 172.21.15.47:0/2919260018 conn(0x7ff18c06e6c0 msgr2=0x7ff18c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 -- 172.21.15.47:0/2919260018 shutdown_connections 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.964+0000 7ff192de9700 1 -- 172.21.15.47:0/2919260018 wait complete. 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.965+0000 7ff192de9700 1 Processor -- start 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.965+0000 7ff192de9700 1 -- start start 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff192de9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 0x7ff18c07b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff192de9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c10ef10 0x7ff18c075780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff192de9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c075cc0 0x7ff18c076170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff192de9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c07e210 con 0x7ff18c10ef10 2024-05-11T12:31:31.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff192de9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c07e090 con 0x7ff18c10b160 2024-05-11T12:31:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff192de9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c07e390 con 0x7ff18c075cc0 2024-05-11T12:31:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff190b85700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 0x7ff18c07b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff190b85700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 0x7ff18c07b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:45960/0 (socket says 172.21.15.47:45960) 2024-05-11T12:31:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff190b85700 1 -- 172.21.15.47:0/2539896009 learned_addr learned my addr 172.21.15.47:0/2539896009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.966+0000 7ff191386700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c075cc0 0x7ff18c076170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.967+0000 7ff190b85700 1 -- 172.21.15.47:0/2539896009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c075cc0 msgr2=0x7ff18c076170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:31.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.967+0000 7ff190b85700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c075cc0 0x7ff18c076170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.967+0000 7ff190b85700 1 -- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c10ef10 msgr2=0x7ff18c075780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.967+0000 7ff190b85700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c10ef10 0x7ff18c075780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.967+0000 7ff190b85700 1 -- 172.21.15.47:0/2539896009 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c138a80 con 0x7ff18c10b160 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.967+0000 7ff190b85700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 0x7ff18c07b4a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00af40 tx=0x7ff17c013530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.968+0000 7ff189ffb700 1 -- 172.21.15.47:0/2539896009 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c022020 con 0x7ff18c10b160 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.968+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18c138d10 con 0x7ff18c10b160 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.968+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18c139250 con 0x7ff18c10b160 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.969+0000 7ff189ffb700 1 -- 172.21.15.47:0/2539896009 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff17c013800 con 0x7ff18c10b160 2024-05-11T12:31:31.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:31.969+0000 7ff189ffb700 1 -- 172.21.15.47:0/2539896009 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c01e470 con 0x7ff18c10b160 2024-05-11T12:31:32.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.001+0000 7ff189ffb700 1 -- 172.21.15.47:0/2539896009 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff17c025020 con 0x7ff18c10b160 2024-05-11T12:31:32.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.003+0000 7ff189ffb700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff1740b2a80 0x7ff1740b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.003+0000 7ff189ffb700 1 -- 172.21.15.47:0/2539896009 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff17c0e2cc0 con 0x7ff18c10b160 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.008+0000 7ff19d67fb80 1 Processor -- start 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.010+0000 7ff19d67fb80 1 -- start start 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.011+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.011+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e10a0 0x55dd612e1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.011+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.011+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dd61233860 con 0x55dd612e1a60 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.011+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dd611d27c0 con 0x55dd612e10a0 2024-05-11T12:31:32.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.011+0000 7ff190b85700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.012+0000 7ff190b85700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612e6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:44700/0 (socket says 172.21.15.47:44700) 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.012+0000 7ff190b85700 1 -- 172.21.15.47:0/1348479285 learned_addr learned my addr 172.21.15.47:0/1348479285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.012+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1348479285 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dd6122f820 con 0x55dd612e6ad0 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.012+0000 7ff191386700 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.013+0000 7ff191386700 1 -- 172.21.15.47:0/1348479285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 msgr2=0x55dd612e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.013+0000 7ff191386700 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.013+0000 7ff191386700 1 -- 172.21.15.47:0/1348479285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e10a0 msgr2=0x55dd612e1520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:31:32.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.013+0000 7ff191386700 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e10a0 0x55dd612e1520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.013+0000 7ff191386700 1 -- 172.21.15.47:0/1348479285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd611808d0 con 0x55dd612e1a60 2024-05-11T12:31:32.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.014+0000 7ff18bfff700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff1740b2a80 0x7ff1740b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.014+0000 7ff191386700 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ed890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff18000ca90 tx=0x7ff18000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5974d94426f63da9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.015+0000 7ff172ffd700 1 -- 172.21.15.47:0/1348479285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18000ebf0 con 0x55dd612e1a60 2024-05-11T12:31:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.015+0000 7ff172ffd700 1 -- 172.21.15.47:0/1348479285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1800044d0 con 0x55dd612e1a60 2024-05-11T12:31:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.015+0000 7ff172ffd700 1 -- 172.21.15.47:0/1348479285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180010430 con 0x55dd612e1a60 2024-05-11T12:31:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.015+0000 7ff190b85700 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:31:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.017+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1348479285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 msgr2=0x55dd612ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:32.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.017+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ed890 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff18000ca90 tx=0x7ff18000cda0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.019+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1348479285 shutdown_connections 2024-05-11T12:31:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.019+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ed890 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.019+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.019+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1348479285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e10a0 0x55dd612e1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.019+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1348479285 >> 172.21.15.47:0/1348479285 conn(0x55dd6120de60 msgr2=0x55dd612db990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:32.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.019+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1348479285 shutdown_connections 2024-05-11T12:31:32.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.019+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1348479285 wait complete. 2024-05-11T12:31:32.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.020+0000 7ff19d67fb80 1 -- start start 2024-05-11T12:31:32.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.021+0000 7ff19d67fb80 1 Processor -- start 2024-05-11T12:31:32.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.022+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.022+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.022+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd612eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.022+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55dd612ede90 con 0x55dd612e1a60 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.022+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55dd61199c30 con 0x55dd612e6ad0 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.022+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55dd612ee010 con 0x55dd612e10a0 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.023+0000 7ff190b85700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.023+0000 7ff190b85700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612ed5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:44714/0 (socket says 172.21.15.47:44714) 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.023+0000 7ff190b85700 1 -- 172.21.15.47:0/128204389 learned_addr learned my addr 172.21.15.47:0/128204389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:32.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.023+0000 7ff191386700 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd612eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.027+0000 7ff18bfff700 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.027+0000 7ff19d67fb80 10 client.0 _ll_register_callbacks cb 0x55dd611cb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.034+0000 7ff191386700 1 -- 172.21.15.47:0/128204389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 msgr2=0x55dd612ed5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.034+0000 7ff191386700 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612ed5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.034+0000 7ff191386700 1 -- 172.21.15.47:0/128204389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 msgr2=0x55dd612ea8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.034+0000 7ff191386700 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ea8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.034+0000 7ff191386700 1 -- 172.21.15.47:0/128204389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd611cf690 con 0x55dd612e6ad0 2024-05-11T12:31:32.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.037+0000 7ff18bfff700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff1740b2a80 0x7ff1740b4f40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff1840095a0 tx=0x7ff18400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:32.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.046+0000 7ff191386700 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd612eadf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1800046a0 tx=0x7ff1800038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1b6ebff39ebd7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:32.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.060+0000 7ff16bfff700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:31:32.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.060+0000 7ff16bfff700 1 -- 172.21.15.47:0/128204389 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180029060 con 0x55dd612e6ad0 2024-05-11T12:31:32.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.060+0000 7ff16bfff700 1 -- 172.21.15.47:0/128204389 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff180005250 con 0x55dd612e6ad0 2024-05-11T12:31:32.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.061+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1800044d0 con 0x55dd612e6ad0 2024-05-11T12:31:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.066+0000 7ff16bfff700 1 -- 172.21.15.47:0/128204389 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff18000ee90 con 0x55dd612e6ad0 2024-05-11T12:31:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.067+0000 7ff19d67fb80 10 client.25271 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:32.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.067+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dd612f8770 con 0x55dd612e6ad0 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff16bfff700 1 -- 172.21.15.47:0/128204389 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff180004c00 con 0x55dd612e6ad0 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff16bfff700 1 -- 172.21.15.47:0/128204389 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff180003ee0 con 0x55dd612e6ad0 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff19d67fb80 10 client.25271 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff19d67fb80 10 client.25271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff19d67fb80 20 client.25271 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd612fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff19d67fb80 4 client.25271 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:31:32.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.071+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff1800044d0 con 0x55dd612fac40 2024-05-11T12:31:32.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.167+0000 7ff190b85700 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd612fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.176+0000 7ff190b85700 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd612fd030 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495af4633f5a9f75 server_cookie=15cb3824ca7981b4 in_seq=0 out_seq=0 2024-05-11T12:31:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.176+0000 7ff16bfff700 10 client.25271 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.185+0000 7ff16bfff700 1 -- 172.21.15.47:0/128204389 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7ff1800044d0 con 0x55dd612fac40 2024-05-11T12:31:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.185+0000 7ff16bfff700 10 client.25271 handle_command_reply: tid=0 2024-05-11T12:31:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.185+0000 7ff19d67fb80 1 client.25271 shutdown 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.186+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 msgr2=0x55dd612eadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.186+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd612eadf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff1800046a0 tx=0x7ff1800038b0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 shutdown_connections 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd612fd030 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd612eadf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff16bfff700 0 client.25271 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612ed5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/128204389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612ea8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 >> 172.21.15.47:0/128204389 conn(0x55dd6120de60 msgr2=0x55dd612d6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 shutdown_connections 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.187+0000 7ff19d67fb80 1 -- 172.21.15.47:0/128204389 wait complete. 2024-05-11T12:31:32.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.188+0000 7ff19d67fb80 20 client.25271 trim_cache size 0 max 16384 2024-05-11T12:31:32.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.242+0000 7ff19d67fb80 1 Processor -- start 2024-05-11T12:31:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.246+0000 7ff19d67fb80 1 -- start start 2024-05-11T12:31:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.247+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e10a0 0x55dd612eaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.247+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e1a60 0x55dd612e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.247+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.247+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff17800b7a0 con 0x55dd612e10a0 2024-05-11T12:31:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.247+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff17800e020 con 0x55dd612e1a60 2024-05-11T12:31:32.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.247+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff17800be10 con 0x55dd612e6ad0 2024-05-11T12:31:32.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.255+0000 7ff190b85700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e10a0 0x55dd612eaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.255+0000 7ff190b85700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e10a0 0x55dd612eaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51512/0 (socket says 172.21.15.47:51512) 2024-05-11T12:31:32.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.255+0000 7ff190b85700 1 -- 172.21.15.47:0/1791630974 learned_addr learned my addr 172.21.15.47:0/1791630974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:32.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.257+0000 7ff190b85700 1 -- 172.21.15.47:0/1791630974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 msgr2=0x55dd612ed0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:31:32.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.257+0000 7ff190b85700 1 --2- 172.21.15.47:0/1791630974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612ed0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.257+0000 7ff190b85700 1 -- 172.21.15.47:0/1791630974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e1a60 msgr2=0x55dd612e62e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:31:32.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.257+0000 7ff190b85700 1 --2- 172.21.15.47:0/1791630974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e1a60 0x55dd612e62e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.257+0000 7ff190b85700 1 -- 172.21.15.47:0/1791630974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd612dc850 con 0x55dd612e10a0 2024-05-11T12:31:32.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.258+0000 7ff190b85700 1 --2- 172.21.15.47:0/1791630974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e10a0 0x55dd612eaf80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0a9ef0 tx=0x7ff17c0e0560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199ac402990a4c23 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:32.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.265+0000 7ff1727fc700 1 -- 172.21.15.47:0/1791630974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0ae5f0 con 0x55dd612e10a0 2024-05-11T12:31:32.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.265+0000 7ff1727fc700 1 -- 172.21.15.47:0/1791630974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff17c0b3010 con 0x55dd612e10a0 2024-05-11T12:31:32.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.267+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1791630974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e10a0 msgr2=0x55dd612eaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:32.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.267+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1791630974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e10a0 0x55dd612eaf80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0a9ef0 tx=0x7ff17c0e0560 comp rx=0 tx=0).stop 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.268+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1791630974 shutdown_connections 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.268+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1791630974 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e6ad0 0x55dd612ed0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.268+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1791630974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e10a0 0x55dd612eaf80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.268+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/1791630974 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e1a60 0x55dd612e62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.268+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1791630974 >> 172.21.15.47:0/1791630974 conn(0x55dd6120de60 msgr2=0x55dd612ea870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.268+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1791630974 shutdown_connections 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.268+0000 7ff19d67fb80 1 -- 172.21.15.47:0/1791630974 wait complete. 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.269+0000 7ff19d67fb80 1 -- start start 2024-05-11T12:31:32.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.269+0000 7ff19d67fb80 1 Processor -- start 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.270+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.270+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612fe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.270+0000 7ff19d67fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd61364160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.270+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff17800c4e0 con 0x55dd612e1a60 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.270+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff17800d950 con 0x55dd612e6ad0 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.270+0000 7ff19d67fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff17800cbb0 con 0x55dd612e10a0 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.270+0000 7ff19d67fb80 10 client.0 _ll_register_callbacks cb 0x7ff18000c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:32.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.271+0000 7ff18bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612fe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.271+0000 7ff18bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612fe4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51518/0 (socket says 172.21.15.47:51518) 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.271+0000 7ff18bfff700 1 -- 172.21.15.47:0/3978558254 learned_addr learned my addr 172.21.15.47:0/3978558254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.272+0000 7ff18bfff700 1 -- 172.21.15.47:0/3978558254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 msgr2=0x55dd612fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.272+0000 7ff18bfff700 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.272+0000 7ff18bfff700 1 -- 172.21.15.47:0/3978558254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 msgr2=0x55dd61364160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.272+0000 7ff18bfff700 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd61364160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.272+0000 7ff18bfff700 1 -- 172.21.15.47:0/3978558254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd613646a0 con 0x55dd612e1a60 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.272+0000 7ff18bfff700 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612fe4d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff18c06ccc0 tx=0x7ff184004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd11f9b77a2f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:32.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.272+0000 7ff16bfff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:31:32.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.273+0000 7ff16bfff700 1 -- 172.21.15.47:0/3978558254 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184025070 con 0x55dd612e1a60 2024-05-11T12:31:32.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.274+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dd61241240 con 0x55dd612e1a60 2024-05-11T12:31:32.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.274+0000 7ff16bfff700 1 -- 172.21.15.47:0/3978558254 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff184004db0 con 0x55dd612e1a60 2024-05-11T12:31:32.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.275+0000 7ff16bfff700 1 -- 172.21.15.47:0/3978558254 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff18402e050 con 0x55dd612e1a60 2024-05-11T12:31:32.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.277+0000 7ff19d67fb80 10 client.15690 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:32.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.277+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dd61365960 con 0x55dd612e1a60 2024-05-11T12:31:32.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.278+0000 7ff16bfff700 1 -- 172.21.15.47:0/3978558254 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dd61241240 con 0x55dd612e1a60 2024-05-11T12:31:32.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.278+0000 7ff16bfff700 1 -- 172.21.15.47:0/3978558254 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff184028020 con 0x55dd612e1a60 2024-05-11T12:31:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.278+0000 7ff19d67fb80 10 client.15690 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.278+0000 7ff19d67fb80 10 client.15690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.278+0000 7ff19d67fb80 20 client.15690 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:32.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.278+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd61368110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.279+0000 7ff19d67fb80 4 client.15690 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:31:32.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.279+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dd6123b2e0 con 0x55dd612fac40 2024-05-11T12:31:32.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.282+0000 7ff190b85700 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd61368110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.287+0000 7ff190b85700 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd61368110 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd89bf74e720c server_cookie=4641f9db91eeda9d in_seq=0 out_seq=0 2024-05-11T12:31:32.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.287+0000 7ff16bfff700 10 client.15690 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.291+0000 7ff16bfff700 1 -- 172.21.15.47:0/3978558254 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dd6123b2e0 con 0x55dd612fac40 2024-05-11T12:31:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.291+0000 7ff16bfff700 10 client.15690 handle_command_reply: tid=0 2024-05-11T12:31:32.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.291+0000 7ff19d67fb80 1 client.15690 shutdown 2024-05-11T12:31:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 msgr2=0x55dd612fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612fe4d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff18c06ccc0 tx=0x7ff184004ad0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 shutdown_connections 2024-05-11T12:31:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55dd612fac40 0x55dd61368110 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55dd612e6ad0 0x55dd61364160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55dd612e10a0 0x55dd612fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 --2- 172.21.15.47:0/3978558254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55dd612e1a60 0x55dd612fe4d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 >> 172.21.15.47:0/3978558254 conn(0x55dd6120de60 msgr2=0x55dd612e0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:32.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff16bfff700 0 client.15690 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.296+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 shutdown_connections 2024-05-11T12:31:32.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.298+0000 7ff19d67fb80 1 -- 172.21.15.47:0/3978558254 wait complete. 2024-05-11T12:31:32.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.298+0000 7ff19d67fb80 20 client.15690 trim_cache size 0 max 16384 2024-05-11T12:31:32.299 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:31:32.299 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:31:32.299 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:31:32.300 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:31:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff1740b2a80 msgr2=0x7ff1740b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff1740b2a80 0x7ff1740b4f40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff1840095a0 tx=0x7ff18400d040 comp rx=0 tx=0).stop 2024-05-11T12:31:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 msgr2=0x7ff18c07b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 0x7ff18c07b4a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00af40 tx=0x7ff17c013530 comp rx=0 tx=0).stop 2024-05-11T12:31:32.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 shutdown_connections 2024-05-11T12:31:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff18c075cc0 0x7ff18c076170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff1740b2a80 0x7ff1740b4f40 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff1840095a0 tx=0x7ff18400d040 comp rx=0 tx=0).stop 2024-05-11T12:31:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff18c10b160 0x7ff18c07b4a0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00af40 tx=0x7ff17c013530 comp rx=0 tx=0).stop 2024-05-11T12:31:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 --2- 172.21.15.47:0/2539896009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff18c10ef10 0x7ff18c075780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:32.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.300+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 >> 172.21.15.47:0/2539896009 conn(0x7ff18c06e6c0 msgr2=0x7ff18c07f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.306+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 shutdown_connections 2024-05-11T12:31:32.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:32.306+0000 7ff192de9700 1 -- 172.21.15.47:0/2539896009 wait complete. 2024-05-11T12:31:33.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:33 smithi064 ceph-mon[34537]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 1.71k op/s 2024-05-11T12:31:33.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:33 smithi050 ceph-mon[34072]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 1.71k op/s 2024-05-11T12:31:33.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:33 smithi047 ceph-mon[30375]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 875 MiB/s wr, 1.71k op/s 2024-05-11T12:31:34.650 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:31:34.650 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:31:35.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:35 smithi047 ceph-mon[30375]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 722 MiB/s wr, 1.45k op/s 2024-05-11T12:31:35.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:35 smithi050 ceph-mon[34072]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 722 MiB/s wr, 1.45k op/s 2024-05-11T12:31:35.790 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:31:35.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:35 smithi064 ceph-mon[34537]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 722 MiB/s wr, 1.45k op/s 2024-05-11T12:31:37.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:37 smithi050 ceph-mon[34072]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 753 MiB/s wr, 1.52k op/s 2024-05-11T12:31:37.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:37 smithi047 ceph-mon[30375]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 753 MiB/s wr, 1.52k op/s 2024-05-11T12:31:37.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:37 smithi064 ceph-mon[34537]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 753 MiB/s wr, 1.52k op/s 2024-05-11T12:31:39.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:39 smithi050 ceph-mon[34072]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.82k op/s 2024-05-11T12:31:39.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:39 smithi047 ceph-mon[30375]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.82k op/s 2024-05-11T12:31:39.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:39 smithi064 ceph-mon[34537]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 873 MiB/s wr, 1.82k op/s 2024-05-11T12:31:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.490+0000 7f256f59e700 1 -- 172.21.15.47:0/2175841338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2570111c40 msgr2=0x7f257010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.490+0000 7f256f59e700 1 --2- 172.21.15.47:0/2175841338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2570111c40 0x7f257010a550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2564009990 tx=0x7f2564009ca0 comp rx=0 tx=0).stop 2024-05-11T12:31:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.495+0000 7f256f59e700 1 -- 172.21.15.47:0/2175841338 shutdown_connections 2024-05-11T12:31:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.495+0000 7f256f59e700 1 --2- 172.21.15.47:0/2175841338 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f257011cd30 0x7f257011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.495+0000 7f256f59e700 1 --2- 172.21.15.47:0/2175841338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2570111c40 0x7f257010a550 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:40.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.495+0000 7f256f59e700 1 --2- 172.21.15.47:0/2175841338 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f25701120b0 0x7f257010aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.495+0000 7f256f59e700 1 -- 172.21.15.47:0/2175841338 >> 172.21.15.47:0/2175841338 conn(0x7f257006e530 msgr2=0x7f257006e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.496+0000 7f256f59e700 1 -- 172.21.15.47:0/2175841338 shutdown_connections 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.498+0000 7f256f59e700 1 -- 172.21.15.47:0/2175841338 wait complete. 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.499+0000 7f256f59e700 1 Processor -- start 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.500+0000 7f256f59e700 1 -- start start 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.500+0000 7f256f59e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570111c40 0x7f257011a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.500+0000 7f256f59e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25701120b0 0x7f2570113350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.500+0000 7f256f59e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f257011cd30 0x7f2570113890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.500+0000 7f256f59e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2570122610 con 0x7f257011cd30 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.500+0000 7f256f59e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2570122490 con 0x7f25701120b0 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.500+0000 7f256f59e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2570122790 con 0x7f2570111c40 2024-05-11T12:31:40.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.502+0000 7f256ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f257011cd30 0x7f2570113890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.503+0000 7f256ed9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f257011cd30 0x7f2570113890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34262/0 (socket says 172.21.15.47:34262) 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.503+0000 7f256ed9d700 1 -- 172.21.15.47:0/4199457834 learned_addr learned my addr 172.21.15.47:0/4199457834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.503+0000 7f256ed9d700 1 -- 172.21.15.47:0/4199457834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570111c40 msgr2=0x7f257011a2c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.503+0000 7f256ed9d700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570111c40 0x7f257011a2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.503+0000 7f256ed9d700 1 -- 172.21.15.47:0/4199457834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25701120b0 msgr2=0x7f2570113350 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.503+0000 7f256ed9d700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25701120b0 0x7f2570113350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.503+0000 7f256ed9d700 1 -- 172.21.15.47:0/4199457834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2570113e80 con 0x7f257011cd30 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.504+0000 7f256ed9d700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f257011cd30 0x7f2570113890 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f256000b450 tx=0x7f256000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.522+0000 7f255f7fe700 1 -- 172.21.15.47:0/4199457834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560015020 con 0x7f257011cd30 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.522+0000 7f255f7fe700 1 -- 172.21.15.47:0/4199457834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2560011ce0 con 0x7f257011cd30 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.522+0000 7f255f7fe700 1 -- 172.21.15.47:0/4199457834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560010d10 con 0x7f257011cd30 2024-05-11T12:31:40.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.524+0000 7f256f59e700 1 -- 172.21.15.47:0/4199457834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2570114170 con 0x7f257011cd30 2024-05-11T12:31:40.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.524+0000 7f256f59e700 1 -- 172.21.15.47:0/4199457834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25701180e0 con 0x7f257011cd30 2024-05-11T12:31:40.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.528+0000 7f256f59e700 1 -- 172.21.15.47:0/4199457834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257006f970 con 0x7f257011cd30 2024-05-11T12:31:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.552+0000 7f255f7fe700 1 -- 172.21.15.47:0/4199457834 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2560018020 con 0x7f257011cd30 2024-05-11T12:31:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.553+0000 7f255f7fe700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f25580b2a90 0x7f25580b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:40.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.554+0000 7f255f7fe700 1 -- 172.21.15.47:0/4199457834 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f25600e2120 con 0x7f257011cd30 2024-05-11T12:31:40.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.559+0000 7f256e59c700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f25580b2a90 0x7f25580b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:40.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.571+0000 7f255f7fe700 1 -- 172.21.15.47:0/4199457834 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25600a9d90 con 0x7f257011cd30 2024-05-11T12:31:40.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:40.575+0000 7f256e59c700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f25580b2a90 0x7f25580b4f50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2564005b40 tx=0x7f2564005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.176+0000 7f256f59e700 1 -- 172.21.15.47:0/4199457834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2570118400 con 0x7f257011cd30 2024-05-11T12:31:41.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.179+0000 7f255f7fe700 1 -- 172.21.15.47:0/4199457834 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f25600a9690 con 0x7f257011cd30 2024-05-11T12:31:41.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:31:41.179 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:31:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.191+0000 7f255d7fa700 1 -- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f25580b2a90 msgr2=0x7f25580b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.191+0000 7f255d7fa700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f25580b2a90 0x7f25580b4f50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2564005b40 tx=0x7f2564005a90 comp rx=0 tx=0).stop 2024-05-11T12:31:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.191+0000 7f255d7fa700 1 -- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f257011cd30 msgr2=0x7f2570113890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.191+0000 7f255d7fa700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f257011cd30 0x7f2570113890 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f256000b450 tx=0x7f256000b810 comp rx=0 tx=0).stop 2024-05-11T12:31:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.195+0000 7f255d7fa700 1 -- 172.21.15.47:0/4199457834 shutdown_connections 2024-05-11T12:31:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.195+0000 7f255d7fa700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f257011cd30 0x7f2570113890 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.195+0000 7f255d7fa700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2570111c40 0x7f257011a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.195+0000 7f255d7fa700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f25580b2a90 0x7f25580b4f50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.195+0000 7f255d7fa700 1 --2- 172.21.15.47:0/4199457834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f25701120b0 0x7f2570113350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.195+0000 7f255d7fa700 1 -- 172.21.15.47:0/4199457834 >> 172.21.15.47:0/4199457834 conn(0x7f257006e530 msgr2=0x7f257010f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.196+0000 7f255d7fa700 1 -- 172.21.15.47:0/4199457834 shutdown_connections 2024-05-11T12:31:41.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:41.196+0000 7f255d7fa700 1 -- 172.21.15.47:0/4199457834 wait complete. 2024-05-11T12:31:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:31:41.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:41 smithi047 ceph-mon[30375]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 793 MiB/s wr, 1.55k op/s 2024-05-11T12:31:41.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:41 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/4199457834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:31:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:41 smithi064 ceph-mon[34537]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 793 MiB/s wr, 1.55k op/s 2024-05-11T12:31:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:41 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/4199457834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:31:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:41 smithi050 ceph-mon[34072]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 793 MiB/s wr, 1.55k op/s 2024-05-11T12:31:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:41 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/4199457834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:31:43.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:43 smithi047 ceph-mon[30375]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 902 MiB/s wr, 1.94k op/s 2024-05-11T12:31:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:43 smithi064 ceph-mon[34537]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 902 MiB/s wr, 1.94k op/s 2024-05-11T12:31:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:43 smithi050 ceph-mon[34072]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 902 MiB/s wr, 1.94k op/s 2024-05-11T12:31:44.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:31:45.366 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:31:45.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:45 smithi047 ceph-mon[30375]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 762 MiB/s wr, 1.77k op/s 2024-05-11T12:31:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:45 smithi064 ceph-mon[34537]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 762 MiB/s wr, 1.77k op/s 2024-05-11T12:31:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:45 smithi050 ceph-mon[34072]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 762 MiB/s wr, 1.77k op/s 2024-05-11T12:31:47.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:47 smithi047 ceph-mon[30375]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 746 MiB/s wr, 1.83k op/s 2024-05-11T12:31:47.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:47 smithi064 ceph-mon[34537]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 746 MiB/s wr, 1.83k op/s 2024-05-11T12:31:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:47 smithi050 ceph-mon[34072]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 746 MiB/s wr, 1.83k op/s 2024-05-11T12:31:48.866 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:48 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:31:48.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:48 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:31:49.041 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:48 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:31:49.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:49 smithi064 ceph-mon[34537]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 896 MiB/s wr, 1.96k op/s 2024-05-11T12:31:49.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:49 smithi050 ceph-mon[34072]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 896 MiB/s wr, 1.96k op/s 2024-05-11T12:31:50.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:49 smithi047 ceph-mon[30375]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 896 MiB/s wr, 1.96k op/s 2024-05-11T12:31:51.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:51 smithi050 ceph-mon[34072]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 759 MiB/s wr, 1.63k op/s 2024-05-11T12:31:52.011 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:51 smithi047 ceph-mon[30375]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 759 MiB/s wr, 1.63k op/s 2024-05-11T12:31:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:51 smithi064 ceph-mon[34537]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 759 MiB/s wr, 1.63k op/s 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.601+0000 7fc2d359e700 1 -- 172.21.15.47:0/1625342261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d4102740 msgr2=0x7fc2d410ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.601+0000 7fc2d359e700 1 --2- 172.21.15.47:0/1625342261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d4102740 0x7fc2d410ea00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c210 tx=0x7fc2cc00c520 comp rx=0 tx=0).stop 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.602+0000 7fc2d359e700 1 -- 172.21.15.47:0/1625342261 shutdown_connections 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.602+0000 7fc2d359e700 1 --2- 172.21.15.47:0/1625342261 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d4102740 0x7fc2d410ea00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.602+0000 7fc2d359e700 1 --2- 172.21.15.47:0/1625342261 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4107be0 0x7fc2d4107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.602+0000 7fc2d359e700 1 --2- 172.21.15.47:0/1625342261 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 0x7fc2d4102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.602+0000 7fc2d359e700 1 -- 172.21.15.47:0/1625342261 >> 172.21.15.47:0/1625342261 conn(0x7fc2d4077700 msgr2=0x7fc2d4077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.602+0000 7fc2d359e700 1 -- 172.21.15.47:0/1625342261 shutdown_connections 2024-05-11T12:31:52.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.602+0000 7fc2d359e700 1 -- 172.21.15.47:0/1625342261 wait complete. 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.613+0000 7fc2d359e700 1 Processor -- start 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.613+0000 7fc2d359e700 1 -- start start 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.614+0000 7fc2d359e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 0x7fc2d4197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.615+0000 7fc2d359e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4102740 0x7fc2d4197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.615+0000 7fc2d359e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d4107be0 0x7fc2d419bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.615+0000 7fc2d359e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d4111560 con 0x7fc2d4101d80 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.615+0000 7fc2d359e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41113e0 con 0x7fc2d4102740 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.615+0000 7fc2d359e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41116e0 con 0x7fc2d4107be0 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.616+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 0x7fc2d4197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.616+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 0x7fc2d4197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47696/0 (socket says 172.21.15.47:47696) 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.616+0000 7fc2d259c700 1 -- 172.21.15.47:0/2623439641 learned_addr learned my addr 172.21.15.47:0/2623439641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.619+0000 7fc2d259c700 1 -- 172.21.15.47:0/2623439641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d4107be0 msgr2=0x7fc2d419bfe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:31:52.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.619+0000 7fc2d259c700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d4107be0 0x7fc2d419bfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.619+0000 7fc2d259c700 1 -- 172.21.15.47:0/2623439641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4102740 msgr2=0x7fc2d4197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.619+0000 7fc2d259c700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4102740 0x7fc2d4197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.619+0000 7fc2d259c700 1 -- 172.21.15.47:0/2623439641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d419c720 con 0x7fc2d4101d80 2024-05-11T12:31:52.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.621+0000 7fc2d259c700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 0x7fc2d4197710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400eab0 tx=0x7fc2c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:52.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.624+0000 7fc2c37fe700 1 -- 172.21.15.47:0/2623439641 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400bbc0 con 0x7fc2d4101d80 2024-05-11T12:31:52.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.625+0000 7fc2c37fe700 1 -- 172.21.15.47:0/2623439641 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c400bd60 con 0x7fc2d4101d80 2024-05-11T12:31:52.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.626+0000 7fc2c37fe700 1 -- 172.21.15.47:0/2623439641 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c401d8f0 con 0x7fc2d4101d80 2024-05-11T12:31:52.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.627+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d419ca10 con 0x7fc2d4101d80 2024-05-11T12:31:52.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.627+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d41a4290 con 0x7fc2d4101d80 2024-05-11T12:31:52.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.644+0000 7fc2c37fe700 1 -- 172.21.15.47:0/2623439641 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2c401da90 con 0x7fc2d4101d80 2024-05-11T12:31:52.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.651+0000 7fc2e343fb80 1 Processor -- start 2024-05-11T12:31:52.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.652+0000 7fc2c37fe700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2bc0b2a10 0x7fc2bc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.653+0000 7fc2e343fb80 1 -- start start 2024-05-11T12:31:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.655+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.655+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8df6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.655+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 0x5576a8df7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.655+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5576a8d44860 con 0x5576a8df2ba0 2024-05-11T12:31:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.655+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5576a8ce37c0 con 0x5576a8df2280 2024-05-11T12:31:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.656+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5576a8d40820 con 0x5576a8df6d50 2024-05-11T12:31:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.656+0000 7fc2d1d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8df6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.657+0000 7fc2c37fe700 1 -- 172.21.15.47:0/2623439641 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc2c40e62f0 con 0x7fc2d4101d80 2024-05-11T12:31:52.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.658+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.658+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:33548/0 (socket says 172.21.15.47:33548) 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.658+0000 7fc2d259c700 1 -- 172.21.15.47:0/151680608 learned_addr learned my addr 172.21.15.47:0/151680608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.659+0000 7fc2d259c700 1 -- 172.21.15.47:0/151680608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 msgr2=0x5576a8df7190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.659+0000 7fc2d259c700 1 --2- 172.21.15.47:0/151680608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 0x5576a8df7190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.659+0000 7fc2d259c700 1 -- 172.21.15.47:0/151680608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 msgr2=0x5576a8df6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.659+0000 7fc2d259c700 1 --2- 172.21.15.47:0/151680608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8df6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.659+0000 7fc2d259c700 1 -- 172.21.15.47:0/151680608 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a8c918d0 con 0x5576a8df2280 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.660+0000 7fc2d259c700 1 --2- 172.21.15.47:0/151680608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df2660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009230 tx=0x7fc2c400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18949c1d26d893e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.661+0000 7fc2c0ff9700 1 -- 172.21.15.47:0/151680608 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40adf40 con 0x5576a8df2280 2024-05-11T12:31:52.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.661+0000 7fc2c0ff9700 1 -- 172.21.15.47:0/151680608 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c40ad840 con 0x5576a8df2280 2024-05-11T12:31:52.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.662+0000 7fc2d2d9d700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2bc0b2a10 0x7fc2bc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.663+0000 7fc2e343fb80 1 -- 172.21.15.47:0/151680608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 msgr2=0x5576a8df2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.663+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/151680608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df2660 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009230 tx=0x7fc2c400bf10 comp rx=0 tx=0).stop 2024-05-11T12:31:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.666+0000 7fc2d2d9d700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2bc0b2a10 0x7fc2bc0b4ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c8f0 tx=0x7fc2cc006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.666+0000 7fc2e343fb80 1 -- 172.21.15.47:0/151680608 shutdown_connections 2024-05-11T12:31:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.666+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/151680608 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 0x5576a8df7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.666+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/151680608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df2660 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.666+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/151680608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8df6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.667+0000 7fc2e343fb80 1 -- 172.21.15.47:0/151680608 >> 172.21.15.47:0/151680608 conn(0x5576a8d1ee60 msgr2=0x5576a8deccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.667+0000 7fc2e343fb80 1 -- 172.21.15.47:0/151680608 shutdown_connections 2024-05-11T12:31:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.668+0000 7fc2e343fb80 1 -- 172.21.15.47:0/151680608 wait complete. 2024-05-11T12:31:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.669+0000 7fc2e343fb80 1 -- start start 2024-05-11T12:31:52.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.669+0000 7fc2e343fb80 1 Processor -- start 2024-05-11T12:31:52.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.670+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2280 0x5576a8dfe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.671+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8dfb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.671+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2280 0x5576a8dfe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.671+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2280 0x5576a8dfe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:58968/0 (socket says 172.21.15.47:58968) 2024-05-11T12:31:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.671+0000 7fc2d259c700 1 -- 172.21.15.47:0/263121114 learned_addr learned my addr 172.21.15.47:0/263121114 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.671+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.672+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5576a8dfef50 con 0x5576a8df2ba0 2024-05-11T12:31:52.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.672+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5576a8caac30 con 0x5576a8df6d50 2024-05-11T12:31:52.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.672+0000 7fc2d2d9d700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.673+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5576a8dff0d0 con 0x5576a8df2280 2024-05-11T12:31:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.673+0000 7fc2e343fb80 10 client.0 _ll_register_callbacks cb 0x5576a8cdc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:52.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.673+0000 7fc2d1d9b700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8dfb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.675+0000 7fc2d1d9b700 1 -- 172.21.15.47:0/263121114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2280 msgr2=0x5576a8dfe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.675+0000 7fc2d1d9b700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2280 0x5576a8dfe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.675+0000 7fc2d1d9b700 1 -- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 msgr2=0x5576a8dfbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.675+0000 7fc2d1d9b700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.675+0000 7fc2d1d9b700 1 -- 172.21.15.47:0/263121114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a8ce0690 con 0x5576a8df2ba0 2024-05-11T12:31:52.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.675+0000 7fc2d1d9b700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8dfb590 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4107580 tx=0x7fc2c800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed01eccc7ee2c4b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:52.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.678+0000 7fc2d259c700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2280 0x5576a8dfe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:31:52.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.678+0000 7fc2d2d9d700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:31:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.683+0000 7fc2b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:31:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.683+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/263121114 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8004220 con 0x5576a8df2ba0 2024-05-11T12:31:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.683+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/263121114 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc2c80043c0 con 0x5576a8df2ba0 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.685+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c40ad840 con 0x5576a8df2ba0 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.685+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/263121114 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc2c801f050 con 0x5576a8df2ba0 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.685+0000 7fc2e343fb80 10 client.15705 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.685+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5576a8e09860 con 0x5576a8df2ba0 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.686+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/263121114 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c40ad840 con 0x5576a8df2ba0 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.687+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/263121114 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc2c8003ad0 con 0x5576a8df2ba0 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.688+0000 7fc2e343fb80 10 client.15705 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.688+0000 7fc2e343fb80 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.688+0000 7fc2e343fb80 20 client.15705 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.688+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0bea0 0x5576a8e0e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.688+0000 7fc2e343fb80 4 client.15705 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:31:52.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.688+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576a8d52240 con 0x5576a8e0bea0 2024-05-11T12:31:52.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.690+0000 7fc2d259c700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0bea0 0x5576a8e0e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.693+0000 7fc2d259c700 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0bea0 0x5576a8e0e290 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccbae9eb53be8311 server_cookie=443a6b589dd29daa in_seq=0 out_seq=0 2024-05-11T12:31:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.693+0000 7fc2b9ffb700 10 client.15705 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.696+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/263121114 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576a8d52240 con 0x5576a8e0bea0 2024-05-11T12:31:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.696+0000 7fc2b9ffb700 10 client.15705 handle_command_reply: tid=0 2024-05-11T12:31:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.696+0000 7fc2e343fb80 1 client.15705 shutdown 2024-05-11T12:31:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.697+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 msgr2=0x5576a8dfb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.697+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8dfb590 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4107580 tx=0x7fc2c800e9c0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 shutdown_connections 2024-05-11T12:31:52.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0bea0 0x5576a8e0e290 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2280 0x5576a8dfe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/263121114 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8dfb590 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 >> 172.21.15.47:0/263121114 conn(0x5576a8d1ee60 msgr2=0x5576a8decbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2b9ffb700 0 client.15705 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 shutdown_connections 2024-05-11T12:31:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 1 -- 172.21.15.47:0/263121114 wait complete. 2024-05-11T12:31:52.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.702+0000 7fc2e343fb80 20 client.15705 trim_cache size 0 max 16384 2024-05-11T12:31:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 Processor -- start 2024-05-11T12:31:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 -- start start 2024-05-11T12:31:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8e07fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 0x5576a8df7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41a99b0 con 0x5576a8df2ba0 2024-05-11T12:31:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41ac180 con 0x5576a8df2280 2024-05-11T12:31:52.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.738+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41a9f70 con 0x5576a8df6d50 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.739+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.739+0000 7fc2d259c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:33572/0 (socket says 172.21.15.47:33572) 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.739+0000 7fc2d259c700 1 -- 172.21.15.47:0/2834330893 learned_addr learned my addr 172.21.15.47:0/2834330893 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.739+0000 7fc2d1d9b700 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8e07fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.740+0000 7fc2d259c700 1 -- 172.21.15.47:0/2834330893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 msgr2=0x5576a8df7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.740+0000 7fc2d259c700 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 0x5576a8df7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.740+0000 7fc2d259c700 1 -- 172.21.15.47:0/2834330893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 msgr2=0x5576a8e07fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.740+0000 7fc2d259c700 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8e07fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.740+0000 7fc2d259c700 1 -- 172.21.15.47:0/2834330893 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a8e17100 con 0x5576a8df2280 2024-05-11T12:31:52.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.742+0000 7fc2d259c700 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df91b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40e4090 tx=0x7fc2c4015f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b46abb84c83658 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.745+0000 7fc2bbfff700 1 -- 172.21.15.47:0/2834330893 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40adf40 con 0x5576a8df2280 2024-05-11T12:31:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.745+0000 7fc2bbfff700 1 -- 172.21.15.47:0/2834330893 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c401dd80 con 0x5576a8df2280 2024-05-11T12:31:52.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.745+0000 7fc2bbfff700 1 -- 172.21.15.47:0/2834330893 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40b6470 con 0x5576a8df2280 2024-05-11T12:31:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.747+0000 7fc2e343fb80 1 -- 172.21.15.47:0/2834330893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 msgr2=0x5576a8df91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.747+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df91b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40e4090 tx=0x7fc2c4015f90 comp rx=0 tx=0).stop 2024-05-11T12:31:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 -- 172.21.15.47:0/2834330893 shutdown_connections 2024-05-11T12:31:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df6d50 0x5576a8df7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df2280 0x5576a8df91b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/2834330893 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2ba0 0x5576a8e07fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 -- 172.21.15.47:0/2834330893 >> 172.21.15.47:0/2834330893 conn(0x5576a8d1ee60 msgr2=0x5576a8df7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:52.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 -- 172.21.15.47:0/2834330893 shutdown_connections 2024-05-11T12:31:52.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 -- 172.21.15.47:0/2834330893 wait complete. 2024-05-11T12:31:52.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 -- start start 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.750+0000 7fc2e343fb80 1 Processor -- start 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2280 0x5576a8e04640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2ba0 0x5576a8dff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2e343fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41aa640 con 0x5576a8df2280 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41abab0 con 0x5576a8df6d50 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2e343fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d41aad10 con 0x5576a8df2ba0 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2e343fb80 10 client.0 _ll_register_callbacks cb 0x5576a8cdc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2d2d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2d2d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:33580/0 (socket says 172.21.15.47:33580) 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.751+0000 7fc2d2d9d700 1 -- 172.21.15.47:0/121435968 learned_addr learned my addr 172.21.15.47:0/121435968 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.752+0000 7fc2d2d9d700 1 -- 172.21.15.47:0/121435968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2ba0 msgr2=0x5576a8dff9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.752+0000 7fc2d2d9d700 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2ba0 0x5576a8dff9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.752+0000 7fc2d2d9d700 1 -- 172.21.15.47:0/121435968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2280 msgr2=0x5576a8e04640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:31:52.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.752+0000 7fc2d2d9d700 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2280 0x5576a8e04640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.752+0000 7fc2d2d9d700 1 -- 172.21.15.47:0/121435968 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a8e004c0 con 0x5576a8df6d50 2024-05-11T12:31:52.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.756+0000 7fc2d2d9d700 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfff00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c210 tx=0x7fc2cc00df00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=912f1e60142c4837 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:31:52.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.757+0000 7fc2b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:31:52.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.757+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/121435968 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc02c020 con 0x5576a8df6d50 2024-05-11T12:31:52.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.757+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/121435968 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc2cc007880 con 0x5576a8df6d50 2024-05-11T12:31:52.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.758+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c401dd80 con 0x5576a8df6d50 2024-05-11T12:31:52.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.762+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/121435968 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc2cc024040 con 0x5576a8df6d50 2024-05-11T12:31:52.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.763+0000 7fc2e343fb80 10 client.25283 fetch_fsmap learned FSMap version 20 2024-05-11T12:31:52.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.763+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5576a8e0f6c0 con 0x5576a8df6d50 2024-05-11T12:31:52.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/121435968 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2c401dd80 con 0x5576a8df6d50 2024-05-11T12:31:52.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/121435968 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc2cc039020 con 0x5576a8df6d50 2024-05-11T12:31:52.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2e343fb80 10 client.25283 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:31:52.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2e343fb80 10 client.25283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:31:52.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2e343fb80 20 client.25283 populate_metadata read hostname 'smithi047' 2024-05-11T12:31:52.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0fe30 0x5576a8e12250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:31:52.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2e343fb80 4 client.25283 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:31:52.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.765+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576a8d4c2e0 con 0x5576a8e0fe30 2024-05-11T12:31:52.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.766+0000 7fc2d259c700 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0fe30 0x5576a8e12250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:31:52.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.767+0000 7fc2d259c700 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0fe30 0x5576a8e12250 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b3a6d0da4b2c22a server_cookie=b6e505dd103691ad in_seq=0 out_seq=0 2024-05-11T12:31:52.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.767+0000 7fc2b9ffb700 10 client.25283 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:52.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.768+0000 7fc2b9ffb700 1 -- 172.21.15.47:0/121435968 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576a8d4c2e0 con 0x5576a8e0fe30 2024-05-11T12:31:52.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.768+0000 7fc2b9ffb700 10 client.25283 handle_command_reply: tid=0 2024-05-11T12:31:52.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.769+0000 7fc2e343fb80 1 client.25283 shutdown 2024-05-11T12:31:52.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.769+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 msgr2=0x5576a8dfff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.769+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfff00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c210 tx=0x7fc2cc00df00 comp rx=0 tx=0).stop 2024-05-11T12:31:52.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 shutdown_connections 2024-05-11T12:31:52.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5576a8e0fe30 0x5576a8e12250 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5576a8df6d50 0x5576a8dfff00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5576a8df2280 0x5576a8e04640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2e343fb80 1 --2- 172.21.15.47:0/121435968 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5576a8df2ba0 0x5576a8dff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 >> 172.21.15.47:0/121435968 conn(0x5576a8d1ee60 msgr2=0x5576a8df1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:52.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2b9ffb700 0 client.25283 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:31:52.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.772+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 shutdown_connections 2024-05-11T12:31:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.774+0000 7fc2e343fb80 1 -- 172.21.15.47:0/121435968 wait complete. 2024-05-11T12:31:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.775+0000 7fc2e343fb80 20 client.25283 trim_cache size 0 max 16384 2024-05-11T12:31:52.777 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:31:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2bc0b2a10 msgr2=0x7fc2bc0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2bc0b2a10 0x7fc2bc0b4ed0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00c8f0 tx=0x7fc2cc006000 comp rx=0 tx=0).stop 2024-05-11T12:31:52.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 msgr2=0x7fc2d4197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 0x7fc2d4197710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400eab0 tx=0x7fc2c400ee70 comp rx=0 tx=0).stop 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 shutdown_connections 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2d4107be0 0x7fc2d419bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2d4101d80 0x7fc2d4197710 secure :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400eab0 tx=0x7fc2c400ee70 comp rx=0 tx=0).stop 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2bc0b2a10 0x7fc2bc0b4ed0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 --2- 172.21.15.47:0/2623439641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2d4102740 0x7fc2d4197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 >> 172.21.15.47:0/2623439641 conn(0x7fc2d4077700 msgr2=0x7fc2d4100510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 shutdown_connections 2024-05-11T12:31:52.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:31:52.777+0000 7fc2d359e700 1 -- 172.21.15.47:0/2623439641 wait complete. 2024-05-11T12:31:52.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:31:52.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:31:52.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:52 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:31:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:31:53.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:31:53.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:52 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:31:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:31:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:31:53.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:52 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:31:53.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:53 smithi050 ceph-mon[34072]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 828 MiB/s wr, 1.72k op/s 2024-05-11T12:31:54.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:53 smithi047 ceph-mon[30375]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 828 MiB/s wr, 1.72k op/s 2024-05-11T12:31:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:53 smithi064 ceph-mon[34537]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 828 MiB/s wr, 1.72k op/s 2024-05-11T12:31:55.885 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:31:55.885 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:31:55.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:55 smithi050 ceph-mon[34072]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 743 MiB/s wr, 1.32k op/s 2024-05-11T12:31:56.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:55 smithi047 ceph-mon[30375]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 743 MiB/s wr, 1.32k op/s 2024-05-11T12:31:56.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:55 smithi064 ceph-mon[34537]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 743 MiB/s wr, 1.32k op/s 2024-05-11T12:31:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:31:57.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:57 smithi050 ceph-mon[34072]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 717 MiB/s wr, 1.17k op/s 2024-05-11T12:31:58.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:57 smithi047 ceph-mon[30375]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 717 MiB/s wr, 1.17k op/s 2024-05-11T12:31:58.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:57 smithi064 ceph-mon[34537]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 717 MiB/s wr, 1.17k op/s 2024-05-11T12:31:59.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:31:59 smithi047 ceph-mon[30375]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 905 MiB/s wr, 1.38k op/s 2024-05-11T12:31:59.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:31:59 smithi050 ceph-mon[34072]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 905 MiB/s wr, 1.38k op/s 2024-05-11T12:32:00.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:31:59 smithi064 ceph-mon[34537]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 905 MiB/s wr, 1.38k op/s 2024-05-11T12:32:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.563+0000 7fc6b62bb700 1 -- 172.21.15.47:0/4253528206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b010aa90 msgr2=0x7fc6b010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.563+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/4253528206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b010aa90 0x7fc6b010af10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b0a0 tx=0x7fc6a800b3b0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.565+0000 7fc6b62bb700 1 -- 172.21.15.47:0/4253528206 shutdown_connections 2024-05-11T12:32:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.565+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/4253528206 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b011cd30 0x7fc6b011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.565+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/4253528206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 0x7fc6b010a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.565+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/4253528206 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b010aa90 0x7fc6b010af10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.565+0000 7fc6b62bb700 1 -- 172.21.15.47:0/4253528206 >> 172.21.15.47:0/4253528206 conn(0x7fc6b006e6c0 msgr2=0x7fc6b006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:01.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.565+0000 7fc6b62bb700 1 -- 172.21.15.47:0/4253528206 shutdown_connections 2024-05-11T12:32:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.565+0000 7fc6b62bb700 1 -- 172.21.15.47:0/4253528206 wait complete. 2024-05-11T12:32:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.567+0000 7fc6b62bb700 1 Processor -- start 2024-05-11T12:32:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.568+0000 7fc6b62bb700 1 -- start start 2024-05-11T12:32:01.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.569+0000 7fc6b62bb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b010aa90 0x7fc6b0116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.569+0000 7fc6b62bb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 0x7fc6b01166a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.569+0000 7fc6b62bb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b011cd30 0x7fc6b011a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.569+0000 7fc6b62bb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b01129e0 con 0x7fc6b0111d30 2024-05-11T12:32:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.569+0000 7fc6b62bb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0112860 con 0x7fc6b010aa90 2024-05-11T12:32:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.569+0000 7fc6b62bb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0112b60 con 0x7fc6b011cd30 2024-05-11T12:32:01.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.570+0000 7fc6af7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 0x7fc6b01166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.570+0000 7fc6af7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 0x7fc6b01166a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43542/0 (socket says 172.21.15.47:43542) 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.570+0000 7fc6af7fe700 1 -- 172.21.15.47:0/1513805498 learned_addr learned my addr 172.21.15.47:0/1513805498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.572+0000 7fc6b4858700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b011cd30 0x7fc6b011a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.572+0000 7fc6af7fe700 1 -- 172.21.15.47:0/1513805498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b011cd30 msgr2=0x7fc6b011a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.572+0000 7fc6af7fe700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b011cd30 0x7fc6b011a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.573+0000 7fc6af7fe700 1 -- 172.21.15.47:0/1513805498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b010aa90 msgr2=0x7fc6b0116160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.573+0000 7fc6af7fe700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b010aa90 0x7fc6b0116160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.573+0000 7fc6af7fe700 1 -- 172.21.15.47:0/1513805498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b011b0c0 con 0x7fc6b0111d30 2024-05-11T12:32:01.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.577+0000 7fc6af7fe700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 0x7fc6b01166a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8006070 tx=0x7fc6a8004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:01.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.580+0000 7fc6ad7fa700 1 -- 172.21.15.47:0/1513805498 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8021070 con 0x7fc6b0111d30 2024-05-11T12:32:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.581+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b011b350 con 0x7fc6b0111d30 2024-05-11T12:32:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.581+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b01bdf40 con 0x7fc6b0111d30 2024-05-11T12:32:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.581+0000 7fc6ad7fa700 1 -- 172.21.15.47:0/1513805498 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a8012d40 con 0x7fc6b0111d30 2024-05-11T12:32:01.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.581+0000 7fc6ad7fa700 1 -- 172.21.15.47:0/1513805498 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8004020 con 0x7fc6b0111d30 2024-05-11T12:32:01.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.601+0000 7fc6ad7fa700 1 -- 172.21.15.47:0/1513805498 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc6a80041e0 con 0x7fc6b0111d30 2024-05-11T12:32:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.603+0000 7fc6ad7fa700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc6980b2a00 0x7fc6980b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.604+0000 7fc6c0b51b80 1 Processor -- start 2024-05-11T12:32:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.604+0000 7fc6c0b51b80 1 -- start start 2024-05-11T12:32:01.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.604+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986bb50 0x56460986bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.604+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x5646098665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.604+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609872910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.605+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5646097b88e0 con 0x564609866120 2024-05-11T12:32:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.605+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564609757840 con 0x56460986bb50 2024-05-11T12:32:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.605+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5646097b48a0 con 0x564609866ae0 2024-05-11T12:32:01.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.606+0000 7fc6ad7fa700 1 -- 172.21.15.47:0/1513805498 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc6a80e3eb0 con 0x7fc6b0111d30 2024-05-11T12:32:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.609+0000 7fc6b4858700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609872910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.610+0000 7fc6affff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986bb50 0x56460986bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.610+0000 7fc6affff700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc6980b2a00 0x7fc6980b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.610+0000 7fc6af7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x5646098665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.611+0000 7fc6b4858700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609872910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:47220/0 (socket says 172.21.15.47:47220) 2024-05-11T12:32:01.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.611+0000 7fc6b4858700 1 -- 172.21.15.47:0/1980416630 learned_addr learned my addr 172.21.15.47:0/1980416630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.613+0000 7fc6affff700 1 -- 172.21.15.47:0/1980416630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 msgr2=0x564609872910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.613+0000 7fc6affff700 1 --2- 172.21.15.47:0/1980416630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609872910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.613+0000 7fc6affff700 1 -- 172.21.15.47:0/1980416630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 msgr2=0x5646098665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.613+0000 7fc6affff700 1 --2- 172.21.15.47:0/1980416630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x5646098665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.614+0000 7fc6affff700 1 -- 172.21.15.47:0/1980416630 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564609705950 con 0x56460986bb50 2024-05-11T12:32:01.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.621+0000 7fc6affff700 1 --2- 172.21.15.47:0/1980416630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986bb50 0x56460986bf30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4001eb0 tx=0x7fc6a4018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a289f1b794156b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:01.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.623+0000 7fc6967fc700 1 -- 172.21.15.47:0/1980416630 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4012040 con 0x56460986bb50 2024-05-11T12:32:01.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.623+0000 7fc6967fc700 1 -- 172.21.15.47:0/1980416630 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a40075d0 con 0x56460986bb50 2024-05-11T12:32:01.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.624+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/1980416630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986bb50 msgr2=0x56460986bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.624+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/1980416630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986bb50 0x56460986bf30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4001eb0 tx=0x7fc6a4018000 comp rx=0 tx=0).stop 2024-05-11T12:32:01.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.625+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/1980416630 shutdown_connections 2024-05-11T12:32:01.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.625+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/1980416630 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609872910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.625+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/1980416630 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986bb50 0x56460986bf30 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4001eb0 tx=0x7fc6a4018000 comp rx=0 tx=0).stop 2024-05-11T12:32:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.625+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/1980416630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x5646098665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.625+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/1980416630 >> 172.21.15.47:0/1980416630 conn(0x564609792ee0 msgr2=0x564609860a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.627+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/1980416630 shutdown_connections 2024-05-11T12:32:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.627+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/1980416630 wait complete. 2024-05-11T12:32:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.628+0000 7fc6c0b51b80 1 -- start start 2024-05-11T12:32:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.628+0000 7fc6c0b51b80 1 Processor -- start 2024-05-11T12:32:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.629+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x564609872710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.630+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x56460986fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.630+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x5646098703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.630+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564609872e50 con 0x56460986ff50 2024-05-11T12:32:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.630+0000 7fc6b4858700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x5646098703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.630+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56460971ecb0 con 0x564609866120 2024-05-11T12:32:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.630+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564609872fd0 con 0x564609866ae0 2024-05-11T12:32:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.630+0000 7fc6b4858700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x5646098703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43548/0 (socket says 172.21.15.47:43548) 2024-05-11T12:32:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.631+0000 7fc6b4858700 1 -- 172.21.15.47:0/2605063873 learned_addr learned my addr 172.21.15.47:0/2605063873 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:01.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.631+0000 7fc6c0b51b80 10 client.0 _ll_register_callbacks cb 0x5646097509a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:01.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.635+0000 7fc6b4858700 1 -- 172.21.15.47:0/2605063873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 msgr2=0x56460986fa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:32:01.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.635+0000 7fc6b4858700 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x56460986fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.635+0000 7fc6b4858700 1 -- 172.21.15.47:0/2605063873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 msgr2=0x564609872710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:32:01.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.635+0000 7fc6b4858700 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x564609872710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.635+0000 7fc6b4858700 1 -- 172.21.15.47:0/2605063873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564609754710 con 0x56460986ff50 2024-05-11T12:32:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.644+0000 7fc6b4858700 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x5646098703d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009ef0 tx=0x7fc6a00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36778c816b759d1a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.644+0000 7fc68f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:32:01.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.644+0000 7fc6affff700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc6980b2a00 0x7fc6980b4ec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009fd0 tx=0x7fc6a4009d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:01.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.645+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2605063873 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0012650 con 0x56460986ff50 2024-05-11T12:32:01.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.645+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2605063873 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc6a00127f0 con 0x56460986ff50 2024-05-11T12:32:01.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.645+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646097c62c0 con 0x56460986ff50 2024-05-11T12:32:01.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.647+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2605063873 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc6a00129f0 con 0x56460986ff50 2024-05-11T12:32:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.650+0000 7fc6c0b51b80 10 client.15717 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:01.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.650+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56460987dd40 con 0x56460986ff50 2024-05-11T12:32:01.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.656+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2605063873 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5646097c62c0 con 0x56460986ff50 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.656+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2605063873 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc6a000f040 con 0x56460986ff50 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.657+0000 7fc6c0b51b80 10 client.15717 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.657+0000 7fc6c0b51b80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.657+0000 7fc6c0b51b80 20 client.15717 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.657+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564609880830 0x564609882cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.657+0000 7fc6c0b51b80 4 client.15717 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.657+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646097c0360 con 0x564609880830 2024-05-11T12:32:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.658+0000 7fc6affff700 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564609880830 0x564609882cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.674+0000 7fc6affff700 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564609880830 0x564609882cd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5503799c5a3d9a8a server_cookie=861e5e86cf287b7e in_seq=0 out_seq=0 2024-05-11T12:32:01.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.674+0000 7fc68f7fe700 10 client.15717 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.678+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2605063873 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646097c0360 con 0x564609880830 2024-05-11T12:32:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.678+0000 7fc68f7fe700 10 client.15717 handle_command_reply: tid=0 2024-05-11T12:32:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.678+0000 7fc6c0b51b80 1 client.15717 shutdown 2024-05-11T12:32:01.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.680+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 msgr2=0x5646098703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.680+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x5646098703d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009ef0 tx=0x7fc6a00116c0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.681+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 shutdown_connections 2024-05-11T12:32:01.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.681+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564609880830 0x564609882cd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.681+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x5646098703d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.681+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x564609872710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.681+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2605063873 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x56460986fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.681+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 >> 172.21.15.47:0/2605063873 conn(0x564609792ee0 msgr2=0x56460985b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.685+0000 7fc68f7fe700 0 client.15717 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.685+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 shutdown_connections 2024-05-11T12:32:01.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.685+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2605063873 wait complete. 2024-05-11T12:32:01.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.687+0000 7fc6c0b51b80 20 client.15717 trim_cache size 0 max 16384 2024-05-11T12:32:01.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.766+0000 7fc6c0b51b80 1 Processor -- start 2024-05-11T12:32:01.770 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:01 smithi047 ceph-mon[30375]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 723 MiB/s wr, 1.17k op/s 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6c0b51b80 1 -- start start 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x56460986b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609865110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x564609865650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c00b830 con 0x56460986ff50 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c00e0b0 con 0x564609866120 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c00bea0 con 0x564609866ae0 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.769+0000 7fc6affff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x56460986b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.770+0000 7fc6affff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x56460986b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:60030/0 (socket says 172.21.15.47:60030) 2024-05-11T12:32:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.770+0000 7fc6affff700 1 -- 172.21.15.47:0/3929873935 learned_addr learned my addr 172.21.15.47:0/3929873935 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.771+0000 7fc6affff700 1 -- 172.21.15.47:0/3929873935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 msgr2=0x564609865110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:32:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.771+0000 7fc6affff700 1 --2- 172.21.15.47:0/3929873935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609865110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.771+0000 7fc6affff700 1 -- 172.21.15.47:0/3929873935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 msgr2=0x564609865650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:32:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.771+0000 7fc6affff700 1 --2- 172.21.15.47:0/3929873935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x564609865650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.772+0000 7fc6affff700 1 -- 172.21.15.47:0/3929873935 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56460985d610 con 0x564609866120 2024-05-11T12:32:01.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.776+0000 7fc6affff700 1 --2- 172.21.15.47:0/3929873935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x56460986b2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006040 tx=0x7fc6a401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c383252a9ffc610 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.783+0000 7fc695ffb700 1 -- 172.21.15.47:0/3929873935 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4030070 con 0x564609866120 2024-05-11T12:32:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.783+0000 7fc695ffb700 1 -- 172.21.15.47:0/3929873935 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a4008a10 con 0x564609866120 2024-05-11T12:32:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.783+0000 7fc695ffb700 1 -- 172.21.15.47:0/3929873935 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4004470 con 0x564609866120 2024-05-11T12:32:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.785+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/3929873935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 msgr2=0x56460986b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.785+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/3929873935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x56460986b2f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4006040 tx=0x7fc6a401f000 comp rx=0 tx=0).stop 2024-05-11T12:32:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.792+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/3929873935 shutdown_connections 2024-05-11T12:32:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.792+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/3929873935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56460986ff50 0x564609865650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.792+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/3929873935 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564609866120 0x56460986b2f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.792+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/3929873935 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609865110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.792+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/3929873935 >> 172.21.15.47:0/3929873935 conn(0x564609792ee0 msgr2=0x56460986abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:01.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.799+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/3929873935 shutdown_connections 2024-05-11T12:32:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.804+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/3929873935 wait complete. 2024-05-11T12:32:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.804+0000 7fc6c0b51b80 1 -- start start 2024-05-11T12:32:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.805+0000 7fc6c0b51b80 1 Processor -- start 2024-05-11T12:32:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.806+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x564609887ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.806+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609888530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.806+0000 7fc6c0b51b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986ff50 0x564609888a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.806+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c00c570 con 0x564609866120 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.806+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c00d9e0 con 0x56460986ff50 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.806+0000 7fc6c0b51b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c00cc40 con 0x564609866ae0 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.807+0000 7fc6c0b51b80 10 client.0 _ll_register_callbacks cb 0x5646097509a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.807+0000 7fc6affff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x564609887ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.807+0000 7fc6affff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x564609887ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43562/0 (socket says 172.21.15.47:43562) 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.807+0000 7fc6affff700 1 -- 172.21.15.47:0/2808707042 learned_addr learned my addr 172.21.15.47:0/2808707042 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.808+0000 7fc6b4858700 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986ff50 0x564609888a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.809+0000 7fc6affff700 1 -- 172.21.15.47:0/2808707042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 msgr2=0x564609888530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:32:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.809+0000 7fc6affff700 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609888530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.809+0000 7fc6affff700 1 -- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986ff50 msgr2=0x564609888a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.809+0000 7fc6affff700 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986ff50 0x564609888a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.809+0000 7fc6affff700 1 -- 172.21.15.47:0/2808707042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646098730a0 con 0x564609866120 2024-05-11T12:32:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.812+0000 7fc6b4858700 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986ff50 0x564609888a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:32:01.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.812+0000 7fc6affff700 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x564609887ff0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40018c0 tx=0x7fc6a4001c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ea5e301bf3247f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:01.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.812+0000 7fc68f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:32:01.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.813+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2808707042 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4030020 con 0x564609866120 2024-05-11T12:32:01.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.813+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2808707042 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc6a4008dc0 con 0x564609866120 2024-05-11T12:32:01.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.815+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a4008a10 con 0x564609866120 2024-05-11T12:32:01.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.816+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2808707042 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc6a4017b20 con 0x564609866120 2024-05-11T12:32:01.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.818+0000 7fc6c0b51b80 10 client.15720 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:01.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.818+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5646098eb520 con 0x564609866120 2024-05-11T12:32:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.822+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2808707042 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a4008a10 con 0x564609866120 2024-05-11T12:32:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.822+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2808707042 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc6a402b440 con 0x564609866120 2024-05-11T12:32:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.823+0000 7fc6c0b51b80 10 client.15720 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:01.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.823+0000 7fc6c0b51b80 10 client.15720 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:01.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.823+0000 7fc6c0b51b80 20 client.15720 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:01.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.823+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5646098eb890 0x5646098edc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:01.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.823+0000 7fc6c0b51b80 4 client.15720 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:32:01.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.823+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5646096382d0 con 0x5646098eb890 2024-05-11T12:32:01.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.826+0000 7fc6af7fe700 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5646098eb890 0x5646098edc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:01.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.829+0000 7fc6af7fe700 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5646098eb890 0x5646098edc80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a13f8cf221a3203 server_cookie=91401664c7fe88bb in_seq=0 out_seq=0 2024-05-11T12:32:01.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.829+0000 7fc68f7fe700 10 client.15720 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:01.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.830+0000 7fc68f7fe700 1 -- 172.21.15.47:0/2808707042 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5646096382d0 con 0x5646098eb890 2024-05-11T12:32:01.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.830+0000 7fc68f7fe700 10 client.15720 handle_command_reply: tid=0 2024-05-11T12:32:01.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.831+0000 7fc6c0b51b80 1 client.15720 shutdown 2024-05-11T12:32:01.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.838+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 msgr2=0x564609887ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.838+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x564609887ff0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc6a40018c0 tx=0x7fc6a4001c20 comp rx=0 tx=0).stop 2024-05-11T12:32:01.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.842+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 shutdown_connections 2024-05-11T12:32:01.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.842+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5646098eb890 0x5646098edc80 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.842+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56460986ff50 0x564609888a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.842+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564609866120 0x564609887ff0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.842+0000 7fc6c0b51b80 1 --2- 172.21.15.47:0/2808707042 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564609866ae0 0x564609888530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.842+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 >> 172.21.15.47:0/2808707042 conn(0x564609792ee0 msgr2=0x564609865110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:01.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.842+0000 7fc68f7fe700 0 client.15720 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:01.849 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:32:01.849 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:32:01.849 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "d7b00a60-4b44-41df-ba54-8470a8f528e3", 2024-05-11T12:32:01.849 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:32:01.849 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.846+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 shutdown_connections 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.846+0000 7fc6c0b51b80 1 -- 172.21.15.47:0/2808707042 wait complete. 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.846+0000 7fc6c0b51b80 20 client.15720 trim_cache size 0 max 16384 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.847+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc6980b2a00 msgr2=0x7fc6980b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.847+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc6980b2a00 0x7fc6980b4ec0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009fd0 tx=0x7fc6a4009d80 comp rx=0 tx=0).stop 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.847+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 msgr2=0x7fc6b01166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.847+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 0x7fc6b01166a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8006070 tx=0x7fc6a8004d90 comp rx=0 tx=0).stop 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.847+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 shutdown_connections 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.847+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc6b011cd30 0x7fc6b011a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.848+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc6980b2a00 0x7fc6980b4ec0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.848+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6b010aa90 0x7fc6b0116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.848+0000 7fc6b62bb700 1 --2- 172.21.15.47:0/1513805498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6b0111d30 0x7fc6b01166a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:01.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.848+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 >> 172.21.15.47:0/1513805498 conn(0x7fc6b006e6c0 msgr2=0x7fc6b011e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:01.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.853+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 shutdown_connections 2024-05-11T12:32:01.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:01.853+0000 7fc6b62bb700 1 -- 172.21.15.47:0/1513805498 wait complete. 2024-05-11T12:32:01.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:01 smithi050 ceph-mon[34072]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 723 MiB/s wr, 1.17k op/s 2024-05-11T12:32:02.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:01 smithi064 ceph-mon[34537]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 723 MiB/s wr, 1.17k op/s 2024-05-11T12:32:02.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:02 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:32:02.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:02 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:02.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:02 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:32:03.015 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:02 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:32:03.015 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:02 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:03.015 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:02 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:32:03.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:02 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:32:03.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:02 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:03.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:02 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:32:03.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:32:03.906 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:03 smithi047 ceph-mon[30375]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 MiB/s wr, 1.38k op/s 2024-05-11T12:32:04.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:03 smithi064 ceph-mon[34537]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 MiB/s wr, 1.38k op/s 2024-05-11T12:32:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:03 smithi050 ceph-mon[34072]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 MiB/s wr, 1.38k op/s 2024-05-11T12:32:04.878 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:32:05.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:05 smithi047 ceph-mon[30375]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 793 MiB/s wr, 1.26k op/s 2024-05-11T12:32:05.977 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:05 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:06.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:05 smithi064 ceph-mon[34537]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 793 MiB/s wr, 1.26k op/s 2024-05-11T12:32:06.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:05 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:05 smithi050 ceph-mon[34072]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 793 MiB/s wr, 1.26k op/s 2024-05-11T12:32:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:05 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:08.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:07 smithi064 ceph-mon[34537]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 719 MiB/s wr, 1.10k op/s 2024-05-11T12:32:08.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:07 smithi050 ceph-mon[34072]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 719 MiB/s wr, 1.10k op/s 2024-05-11T12:32:08.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:07 smithi047 ceph-mon[30375]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 719 MiB/s wr, 1.10k op/s 2024-05-11T12:32:09.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:08 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:32:09.129 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:08 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:32:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:08 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:32:10.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:09 smithi064 ceph-mon[34537]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 920 MiB/s wr, 1.39k op/s 2024-05-11T12:32:10.143 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:09 smithi047 ceph-mon[30375]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 920 MiB/s wr, 1.39k op/s 2024-05-11T12:32:10.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:09 smithi050 ceph-mon[34072]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 920 MiB/s wr, 1.39k op/s 2024-05-11T12:32:10.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.454+0000 7fe40ea97700 1 -- 172.21.15.47:0/758633304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408102d90 msgr2=0x7fe40810f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.454+0000 7fe40ea97700 1 --2- 172.21.15.47:0/758633304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408102d90 0x7fe40810f2a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe400009f90 tx=0x7fe40000b330 comp rx=0 tx=0).stop 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.454+0000 7fe40ea97700 1 -- 172.21.15.47:0/758633304 shutdown_connections 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.454+0000 7fe40ea97700 1 --2- 172.21.15.47:0/758633304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408102d90 0x7fe40810f2a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.454+0000 7fe40ea97700 1 --2- 172.21.15.47:0/758633304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe408108460 0x7fe408108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.454+0000 7fe40ea97700 1 --2- 172.21.15.47:0/758633304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4081023d0 0x7fe408102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.454+0000 7fe40ea97700 1 -- 172.21.15.47:0/758633304 >> 172.21.15.47:0/758633304 conn(0x7fe4080fdee0 msgr2=0x7fe408100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.455+0000 7fe40ea97700 1 -- 172.21.15.47:0/758633304 shutdown_connections 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.455+0000 7fe40ea97700 1 -- 172.21.15.47:0/758633304 wait complete. 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.456+0000 7fe40ea97700 1 Processor -- start 2024-05-11T12:32:10.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.456+0000 7fe40ea97700 1 -- start start 2024-05-11T12:32:10.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.457+0000 7fe40ea97700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4081023d0 0x7fe40819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.457+0000 7fe40ea97700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe408102d90 0x7fe40819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.457+0000 7fe40da95700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4081023d0 0x7fe40819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40da95700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4081023d0 0x7fe40819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:47242/0 (socket says 172.21.15.47:47242) 2024-05-11T12:32:10.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40da95700 1 -- 172.21.15.47:0/1734233363 learned_addr learned my addr 172.21.15.47:0/1734233363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:10.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40ea97700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408108460 0x7fe408198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40ea97700 1 -- 172.21.15.47:0/1734233363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe4081125e0 con 0x7fe408102d90 2024-05-11T12:32:10.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40ea97700 1 -- 172.21.15.47:0/1734233363 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe408112460 con 0x7fe4081023d0 2024-05-11T12:32:10.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40ea97700 1 -- 172.21.15.47:0/1734233363 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe408112760 con 0x7fe408108460 2024-05-11T12:32:10.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40e296700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408108460 0x7fe408198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.458+0000 7fe40d294700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe408102d90 0x7fe40819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.459+0000 7fe40d294700 1 -- 172.21.15.47:0/1734233363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408108460 msgr2=0x7fe408198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.459+0000 7fe40d294700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408108460 0x7fe408198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.459+0000 7fe40d294700 1 -- 172.21.15.47:0/1734233363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4081023d0 msgr2=0x7fe40819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.459+0000 7fe40d294700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4081023d0 0x7fe40819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.459+0000 7fe40d294700 1 -- 172.21.15.47:0/1734233363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4081990c0 con 0x7fe408102d90 2024-05-11T12:32:10.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.459+0000 7fe40d294700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe408102d90 0x7fe40819ec90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009880 tx=0x7fe3f8009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:10.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.463+0000 7fe3feffd700 1 -- 172.21.15.47:0/1734233363 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f800e2d0 con 0x7fe408102d90 2024-05-11T12:32:10.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.463+0000 7fe3feffd700 1 -- 172.21.15.47:0/1734233363 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f8015300 con 0x7fe408102d90 2024-05-11T12:32:10.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.463+0000 7fe40ea97700 1 -- 172.21.15.47:0/1734233363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe408199350 con 0x7fe408102d90 2024-05-11T12:32:10.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.463+0000 7fe40ea97700 1 -- 172.21.15.47:0/1734233363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40806bac0 con 0x7fe408102d90 2024-05-11T12:32:10.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.463+0000 7fe3feffd700 1 -- 172.21.15.47:0/1734233363 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80038e0 con 0x7fe408102d90 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.478+0000 7fe41a58fb80 1 Processor -- start 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.479+0000 7fe41a58fb80 1 -- start start 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.479+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 0x5653de7d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653de71e860 con 0x5653de7d1bd0 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5653de6bd7c0 con 0x5653de7d24f0 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5653de71a820 con 0x5653de7cc6c0 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe40da95700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe40da95700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:47248/0 (socket says 172.21.15.47:47248) 2024-05-11T12:32:10.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.480+0000 7fe40da95700 1 -- 172.21.15.47:0/1124238908 learned_addr learned my addr 172.21.15.47:0/1124238908 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:10.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.481+0000 7fe40d294700 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.483+0000 7fe40e296700 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 0x5653de7d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.484+0000 7fe40d294700 1 -- 172.21.15.47:0/1124238908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 msgr2=0x5653de7cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.484+0000 7fe40d294700 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.484+0000 7fe40d294700 1 -- 172.21.15.47:0/1124238908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 msgr2=0x5653de7d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.484+0000 7fe40d294700 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 0x5653de7d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.484+0000 7fe40d294700 1 -- 172.21.15.47:0/1124238908 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653de66b8d0 con 0x5653de7cc6c0 2024-05-11T12:32:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.485+0000 7fe40d294700 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7ccb20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe40806bce0 tx=0x7fe3f8005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1746d24b7815c5da server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:10.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.487+0000 7fe40ea97700 1 -- 172.21.15.47:0/1124238908 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8035020 con 0x5653de7cc6c0 2024-05-11T12:32:10.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.488+0000 7fe40ea97700 1 -- 172.21.15.47:0/1124238908 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f8007890 con 0x5653de7cc6c0 2024-05-11T12:32:10.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.489+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1124238908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 msgr2=0x5653de7ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.489+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7ccb20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe40806bce0 tx=0x7fe3f8005320 comp rx=0 tx=0).stop 2024-05-11T12:32:10.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.494+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1124238908 shutdown_connections 2024-05-11T12:32:10.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.494+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7ccb20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.494+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 0x5653de7d1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.494+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1124238908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.494+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1124238908 >> 172.21.15.47:0/1124238908 conn(0x5653de6f8e60 msgr2=0x5653de7c6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:10.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.499+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1124238908 shutdown_connections 2024-05-11T12:32:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.501+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1124238908 wait complete. 2024-05-11T12:32:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.501+0000 7fe41a58fb80 1 -- start start 2024-05-11T12:32:10.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.502+0000 7fe41a58fb80 1 Processor -- start 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.503+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7d81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.503+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 0x5653de7d5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.503+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.503+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5653de7d8f50 con 0x5653de7d1bd0 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.503+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5653de684c30 con 0x5653de7d24f0 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.503+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5653de7d90d0 con 0x5653de7cc6c0 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.503+0000 7fe41a58fb80 10 client.0 _ll_register_callbacks cb 0x5653de6b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.504+0000 7fe40e296700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7d81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.504+0000 7fe40e296700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7d81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53850/0 (socket says 172.21.15.47:53850) 2024-05-11T12:32:10.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.504+0000 7fe40e296700 1 -- 172.21.15.47:0/1305953867 learned_addr learned my addr 172.21.15.47:0/1305953867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.510+0000 7fe40e296700 1 -- 172.21.15.47:0/1305953867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 msgr2=0x5653de7d5a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:32:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.510+0000 7fe40e296700 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7d5a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.510+0000 7fe40e296700 1 -- 172.21.15.47:0/1305953867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 msgr2=0x5653de7d5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.510+0000 7fe40e296700 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 0x5653de7d5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.510+0000 7fe40e296700 1 -- 172.21.15.47:0/1305953867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653de6ba690 con 0x5653de7cc6c0 2024-05-11T12:32:10.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.513+0000 7fe40e296700 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7d81a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe40000bed0 tx=0x7fe400003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14713433d671d54 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.514+0000 7fe3f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:32:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.514+0000 7fe3f17fa700 1 -- 172.21.15.47:0/1305953867 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe400007420 con 0x5653de7cc6c0 2024-05-11T12:32:10.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.514+0000 7fe3f17fa700 1 -- 172.21.15.47:0/1305953867 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fe4000075c0 con 0x5653de7cc6c0 2024-05-11T12:32:10.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.517+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f8007890 con 0x5653de7cc6c0 2024-05-11T12:32:10.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.517+0000 7fe3feffd700 1 -- 172.21.15.47:0/1734233363 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe3f8038020 con 0x7fe408102d90 2024-05-11T12:32:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.518+0000 7fe3feffd700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe3f40b2a70 0x7fe3f40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.519+0000 7fe3f17fa700 1 -- 172.21.15.47:0/1305953867 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe400007800 con 0x5653de7cc6c0 2024-05-11T12:32:10.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.519+0000 7fe40da95700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe3f40b2a70 0x7fe3f40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.521+0000 7fe40da95700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe3f40b2a70 0x7fe3f40b4f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe404005fd0 tx=0x7fe404005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.521+0000 7fe41a58fb80 10 client.25342 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:10.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.521+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5653de7f4790 con 0x5653de7cc6c0 2024-05-11T12:32:10.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.526+0000 7fe3feffd700 1 -- 172.21.15.47:0/1734233363 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fe3f80b0080 con 0x7fe408102d90 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.528+0000 7fe3f17fa700 1 -- 172.21.15.47:0/1305953867 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f8007890 con 0x5653de7cc6c0 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.530+0000 7fe3f17fa700 1 -- 172.21.15.47:0/1305953867 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe400030020 con 0x5653de7cc6c0 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.530+0000 7fe41a58fb80 10 client.25342 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.530+0000 7fe41a58fb80 10 client.25342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.530+0000 7fe41a58fb80 20 client.25342 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.530+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de7f7090 0x5653de7f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.530+0000 7fe41a58fb80 4 client.25342 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:32:10.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.530+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653de72c240 con 0x5653de7f7090 2024-05-11T12:32:10.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.547+0000 7fe40d294700 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de7f7090 0x5653de7f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.570+0000 7fe40d294700 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de7f7090 0x5653de7f94d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363dd6d2d2ceccee server_cookie=50d5332576ce52da in_seq=0 out_seq=0 2024-05-11T12:32:10.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.572+0000 7fe3f17fa700 10 client.25342 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.574+0000 7fe3f17fa700 1 -- 172.21.15.47:0/1305953867 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653de72c240 con 0x5653de7f7090 2024-05-11T12:32:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.574+0000 7fe3f17fa700 10 client.25342 handle_command_reply: tid=0 2024-05-11T12:32:10.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.574+0000 7fe41a58fb80 1 client.25342 shutdown 2024-05-11T12:32:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.575+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 msgr2=0x5653de7d81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.575+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7d81a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe40000bed0 tx=0x7fe400003a00 comp rx=0 tx=0).stop 2024-05-11T12:32:10.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.578+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 shutdown_connections 2024-05-11T12:32:10.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.578+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de7f7090 0x5653de7f94d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.578+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d24f0 0x5653de7d5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.578+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7d81a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.578+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/1305953867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d1bd0 0x5653de7d5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.578+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 >> 172.21.15.47:0/1305953867 conn(0x5653de6f8e60 msgr2=0x5653de7cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:10.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.579+0000 7fe3f17fa700 0 client.25342 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:10.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.579+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 shutdown_connections 2024-05-11T12:32:10.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.598+0000 7fe41a58fb80 1 -- 172.21.15.47:0/1305953867 wait complete. 2024-05-11T12:32:10.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.599+0000 7fe41a58fb80 20 client.25342 trim_cache size 0 max 16384 2024-05-11T12:32:10.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.673+0000 7fe41a58fb80 1 Processor -- start 2024-05-11T12:32:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.678+0000 7fe41a58fb80 1 -- start start 2024-05-11T12:32:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.679+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7cc6c0 0x5653de7cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.679+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.679+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7d24f0 0x5653de7fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.679+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe4080714e0 con 0x5653de7cc6c0 2024-05-11T12:32:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.679+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe408073c80 con 0x5653de7d1bd0 2024-05-11T12:32:10.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.679+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe408071a70 con 0x5653de7d24f0 2024-05-11T12:32:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.680+0000 7fe40d294700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.680+0000 7fe40d294700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:47268/0 (socket says 172.21.15.47:47268) 2024-05-11T12:32:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.680+0000 7fe40d294700 1 -- 172.21.15.47:0/4155754374 learned_addr learned my addr 172.21.15.47:0/4155754374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.681+0000 7fe40da95700 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7cc6c0 0x5653de7cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.681+0000 7fe40e296700 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7d24f0 0x5653de7fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.682+0000 7fe40d294700 1 -- 172.21.15.47:0/4155754374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7d24f0 msgr2=0x5653de7fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.682+0000 7fe40d294700 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7d24f0 0x5653de7fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.682+0000 7fe40d294700 1 -- 172.21.15.47:0/4155754374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7cc6c0 msgr2=0x5653de7cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.682+0000 7fe40d294700 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7cc6c0 0x5653de7cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.682+0000 7fe40d294700 1 -- 172.21.15.47:0/4155754374 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653de7c7a50 con 0x5653de7d1bd0 2024-05-11T12:32:10.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.683+0000 7fe40d294700 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7c70d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80074a0 tx=0x7fe3f800e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e18cc3a43ad06e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.690+0000 7fe3f37fe700 1 -- 172.21.15.47:0/4155754374 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80cd070 con 0x5653de7d1bd0 2024-05-11T12:32:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.690+0000 7fe3f37fe700 1 -- 172.21.15.47:0/4155754374 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f80c8e50 con 0x5653de7d1bd0 2024-05-11T12:32:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.690+0000 7fe3f37fe700 1 -- 172.21.15.47:0/4155754374 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8035070 con 0x5653de7d1bd0 2024-05-11T12:32:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.692+0000 7fe41a58fb80 1 -- 172.21.15.47:0/4155754374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 msgr2=0x5653de7c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.692+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7c70d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe3f80074a0 tx=0x7fe3f800e040 comp rx=0 tx=0).stop 2024-05-11T12:32:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.701+0000 7fe41a58fb80 1 -- 172.21.15.47:0/4155754374 shutdown_connections 2024-05-11T12:32:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.701+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7d24f0 0x5653de7fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.701+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7cc6c0 0x5653de7cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.701+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/4155754374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7c70d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.701+0000 7fe41a58fb80 1 -- 172.21.15.47:0/4155754374 >> 172.21.15.47:0/4155754374 conn(0x5653de6f8e60 msgr2=0x5653de7c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:10.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.704+0000 7fe41a58fb80 1 -- 172.21.15.47:0/4155754374 shutdown_connections 2024-05-11T12:32:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.709+0000 7fe41a58fb80 1 -- 172.21.15.47:0/4155754374 wait complete. 2024-05-11T12:32:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.710+0000 7fe41a58fb80 1 -- start start 2024-05-11T12:32:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.710+0000 7fe41a58fb80 1 Processor -- start 2024-05-11T12:32:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.712+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.712+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.712+0000 7fe41a58fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d24f0 0x5653de7dde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.712+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe408072140 con 0x5653de7d24f0 2024-05-11T12:32:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.713+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe4080735b0 con 0x5653de7d1bd0 2024-05-11T12:32:10.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.713+0000 7fe41a58fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe408072810 con 0x5653de7cc6c0 2024-05-11T12:32:10.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.713+0000 7fe40d294700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.713+0000 7fe40d294700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7dd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:47278/0 (socket says 172.21.15.47:47278) 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.713+0000 7fe40d294700 1 -- 172.21.15.47:0/3570029969 learned_addr learned my addr 172.21.15.47:0/3570029969 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.715+0000 7fe40da95700 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.715+0000 7fe41a58fb80 10 client.0 _ll_register_callbacks cb 0x5653de6b6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.716+0000 7fe40d294700 1 -- 172.21.15.47:0/3570029969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 msgr2=0x5653de7dd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.716+0000 7fe40d294700 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7dd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.716+0000 7fe40d294700 1 -- 172.21.15.47:0/3570029969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d24f0 msgr2=0x5653de7dde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.716+0000 7fe40d294700 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d24f0 0x5653de7dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.716+0000 7fe40d294700 1 -- 172.21.15.47:0/3570029969 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653de7de550 con 0x5653de7d1bd0 2024-05-11T12:32:10.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.716+0000 7fe40da95700 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7dd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:32:10.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.720+0000 7fe40d294700 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7dd8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009fd0 tx=0x7fe3f80b4240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13be8ca1b70be2bc server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.725+0000 7fe3f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:32:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.725+0000 7fe3f17fa700 1 -- 172.21.15.47:0/3570029969 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80cd070 con 0x5653de7d1bd0 2024-05-11T12:32:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.725+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3f80c8e50 con 0x5653de7d1bd0 2024-05-11T12:32:10.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.725+0000 7fe3f17fa700 1 -- 172.21.15.47:0/3570029969 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fe3f8004300 con 0x5653de7d1bd0 2024-05-11T12:32:10.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.732+0000 7fe3f17fa700 1 -- 172.21.15.47:0/3570029969 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe3f8035040 con 0x5653de7d1bd0 2024-05-11T12:32:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.736+0000 7fe41a58fb80 10 client.25310 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:10.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.736+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5653de8731d0 con 0x5653de7d1bd0 2024-05-11T12:32:10.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.747+0000 7fe3f17fa700 1 -- 172.21.15.47:0/3570029969 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f80c8e50 con 0x5653de7d1bd0 2024-05-11T12:32:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.748+0000 7fe3f17fa700 1 -- 172.21.15.47:0/3570029969 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fe3f80cf070 con 0x5653de7d1bd0 2024-05-11T12:32:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.749+0000 7fe41a58fb80 10 client.25310 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.749+0000 7fe41a58fb80 10 client.25310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.749+0000 7fe41a58fb80 20 client.25310 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:10.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.749+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de8738c0 0x5653de875ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.749+0000 7fe41a58fb80 4 client.25310 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:32:10.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.749+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653de7262e0 con 0x5653de8738c0 2024-05-11T12:32:10.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.750+0000 7fe40da95700 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de8738c0 0x5653de875ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.754+0000 7fe40da95700 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de8738c0 0x5653de875ce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01356828adf2f25 server_cookie=e1674ac903243272 in_seq=0 out_seq=0 2024-05-11T12:32:10.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.754+0000 7fe3f17fa700 10 client.25310 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:10.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.758+0000 7fe3f17fa700 1 -- 172.21.15.47:0/3570029969 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5653de7262e0 con 0x5653de8738c0 2024-05-11T12:32:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.758+0000 7fe3f17fa700 10 client.25310 handle_command_reply: tid=0 2024-05-11T12:32:10.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.758+0000 7fe41a58fb80 1 client.25310 shutdown 2024-05-11T12:32:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.759+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 msgr2=0x5653de7dd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.759+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7dd8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009fd0 tx=0x7fe3f80b4240 comp rx=0 tx=0).stop 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 shutdown_connections 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5653de8738c0 0x5653de875ce0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5653de7d24f0 0x5653de7dde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5653de7cc6c0 0x5653de7dd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe41a58fb80 1 --2- 172.21.15.47:0/3570029969 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5653de7d1bd0 0x5653de7dd8f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe3f17fa700 0 client.25310 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 >> 172.21.15.47:0/3570029969 conn(0x5653de6f8e60 msgr2=0x5653de7d28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.762+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 shutdown_connections 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.766+0000 7fe41a58fb80 1 -- 172.21.15.47:0/3570029969 wait complete. 2024-05-11T12:32:10.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.766+0000 7fe41a58fb80 20 client.25310 trim_cache size 0 max 16384 2024-05-11T12:32:10.769 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:32:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:32:10.769 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:32:10.769 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:32:10.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.784+0000 7fe3fcff9700 1 -- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe3f40b2a70 msgr2=0x7fe3f40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.784+0000 7fe3fcff9700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe3f40b2a70 0x7fe3f40b4f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe404005fd0 tx=0x7fe404005ee0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.784+0000 7fe3fcff9700 1 -- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe408102d90 msgr2=0x7fe40819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:10.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.784+0000 7fe3fcff9700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe408102d90 0x7fe40819ec90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009880 tx=0x7fe3f8009c40 comp rx=0 tx=0).stop 2024-05-11T12:32:10.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.789+0000 7fe3fcff9700 1 -- 172.21.15.47:0/1734233363 shutdown_connections 2024-05-11T12:32:10.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.789+0000 7fe3fcff9700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe408108460 0x7fe408198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.789+0000 7fe3fcff9700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fe3f40b2a70 0x7fe3f40b4f30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.789+0000 7fe3fcff9700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe4081023d0 0x7fe40819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.789+0000 7fe3fcff9700 1 --2- 172.21.15.47:0/1734233363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe408102d90 0x7fe40819ec90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:10.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.789+0000 7fe3fcff9700 1 -- 172.21.15.47:0/1734233363 >> 172.21.15.47:0/1734233363 conn(0x7fe4080fdee0 msgr2=0x7fe4080fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:10.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.790+0000 7fe3fcff9700 1 -- 172.21.15.47:0/1734233363 shutdown_connections 2024-05-11T12:32:10.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:10.793+0000 7fe3fcff9700 1 -- 172.21.15.47:0/1734233363 wait complete. 2024-05-11T12:32:12.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:11 smithi047 ceph-mon[30375]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 746 MiB/s wr, 1.15k op/s 2024-05-11T12:32:12.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:11 smithi064 ceph-mon[34537]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 746 MiB/s wr, 1.15k op/s 2024-05-11T12:32:12.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:11 smithi050 ceph-mon[34072]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 746 MiB/s wr, 1.15k op/s 2024-05-11T12:32:12.889 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:32:12.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:32:13.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:12 smithi047 ceph-mon[30375]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 874 MiB/s wr, 1.41k op/s 2024-05-11T12:32:13.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:12 smithi050 ceph-mon[34072]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 874 MiB/s wr, 1.41k op/s 2024-05-11T12:32:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:12 smithi064 ceph-mon[34537]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 874 MiB/s wr, 1.41k op/s 2024-05-11T12:32:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:32:15.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:15 smithi047 ceph-mon[30375]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 804 MiB/s wr, 1.32k op/s 2024-05-11T12:32:15.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:15 smithi064 ceph-mon[34537]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 804 MiB/s wr, 1.32k op/s 2024-05-11T12:32:15.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:15 smithi050 ceph-mon[34072]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 804 MiB/s wr, 1.32k op/s 2024-05-11T12:32:17.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:17 smithi047 ceph-mon[30375]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 746 MiB/s wr, 1.23k op/s 2024-05-11T12:32:17.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:17 smithi064 ceph-mon[34537]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 746 MiB/s wr, 1.23k op/s 2024-05-11T12:32:17.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:17 smithi050 ceph-mon[34072]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 746 MiB/s wr, 1.23k op/s 2024-05-11T12:32:19.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:19 smithi047 ceph-mon[30375]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 947 MiB/s wr, 1.57k op/s 2024-05-11T12:32:19.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:19 smithi064 ceph-mon[34537]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 947 MiB/s wr, 1.57k op/s 2024-05-11T12:32:19.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:19 smithi050 ceph-mon[34072]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 947 MiB/s wr, 1.57k op/s 2024-05-11T12:32:20.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.981+0000 7fc2f81cf700 1 -- 172.21.15.47:0/301892561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010aa90 msgr2=0x7fc2f010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.981+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/301892561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010aa90 0x7fc2f010af10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009990 tx=0x7fc2e4009ca0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.981+0000 7fc2f81cf700 1 -- 172.21.15.47:0/301892561 shutdown_connections 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.981+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/301892561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2f011cd30 0x7fc2f011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.981+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/301892561 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2f0111d30 0x7fc2f010a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.981+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/301892561 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f010aa90 0x7fc2f010af10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.981+0000 7fc2f81cf700 1 -- 172.21.15.47:0/301892561 >> 172.21.15.47:0/301892561 conn(0x7fc2f006e6c0 msgr2=0x7fc2f006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.985+0000 7fc2f81cf700 1 -- 172.21.15.47:0/301892561 shutdown_connections 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.985+0000 7fc2f81cf700 1 -- 172.21.15.47:0/301892561 wait complete. 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.986+0000 7fc2f81cf700 1 Processor -- start 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.986+0000 7fc2f81cf700 1 -- start start 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.986+0000 7fc2f81cf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2f010aa90 0x7fc2f011c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:20.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.986+0000 7fc2f81cf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f0111d30 0x7fc2f011c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.986+0000 7fc2f81cf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2f011cd30 0x7fc2f01160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.987+0000 7fc2f81cf700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0112b20 con 0x7fc2f010aa90 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.987+0000 7fc2f81cf700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01129a0 con 0x7fc2f0111d30 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.987+0000 7fc2f81cf700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0112ca0 con 0x7fc2f011cd30 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.987+0000 7fc2f676c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2f011cd30 0x7fc2f01160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.987+0000 7fc2f576a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f0111d30 0x7fc2f011c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.988+0000 7fc2f5f6b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2f010aa90 0x7fc2f011c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.988+0000 7fc2f676c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2f011cd30 0x7fc2f01160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:60590/0 (socket says 172.21.15.47:60590) 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.988+0000 7fc2f676c700 1 -- 172.21.15.47:0/572595387 learned_addr learned my addr 172.21.15.47:0/572595387 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2f5f6b700 1 -- 172.21.15.47:0/572595387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2f011cd30 msgr2=0x7fc2f01160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2f5f6b700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2f011cd30 0x7fc2f01160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2f5f6b700 1 -- 172.21.15.47:0/572595387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f0111d30 msgr2=0x7fc2f011c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2f5f6b700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f0111d30 0x7fc2f011c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2f5f6b700 1 -- 172.21.15.47:0/572595387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f0116910 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2f576a700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f0111d30 0x7fc2f011c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2f5f6b700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2f010aa90 0x7fc2f011c000 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d950 tx=0x7fc2e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.989+0000 7fc2eeffd700 1 -- 172.21.15.47:0/572595387 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8016070 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.990+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f0116c00 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.990+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01be060 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.990+0000 7fc2eeffd700 1 -- 172.21.15.47:0/572595387 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e800fd40 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.990+0000 7fc2eeffd700 1 -- 172.21.15.47:0/572595387 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e800e3b0 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:19.995+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2dc005330 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.002+0000 7fc2eeffd700 1 -- 172.21.15.47:0/572595387 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc2e8019070 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.003+0000 7fc2eeffd700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2d80b2940 0x7fc2d80b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.003+0000 7fc2eeffd700 1 -- 172.21.15.47:0/572595387 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc2e80e1b60 con 0x7fc2f010aa90 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.008+0000 7fc2f576a700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2d80b2940 0x7fc2d80b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:20.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.012+0000 7fc2f576a700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2d80b2940 0x7fc2d80b4e00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0117840 tx=0x7fc2e4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:20.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.019+0000 7fc2eeffd700 1 -- 172.21.15.47:0/572595387 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2e80a97b0 con 0x7fc2f010aa90 2024-05-11T12:32:20.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.703+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2dc005620 con 0x7fc2f010aa90 2024-05-11T12:32:20.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.706+0000 7fc2eeffd700 1 -- 172.21.15.47:0/572595387 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fc2e80a90b0 con 0x7fc2f010aa90 2024-05-11T12:32:20.718 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:32:20.718 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2d80b2940 msgr2=0x7fc2d80b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2d80b2940 0x7fc2d80b4e00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0117840 tx=0x7fc2e4005fd0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2f010aa90 msgr2=0x7fc2f011c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2f010aa90 0x7fc2f011c000 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d950 tx=0x7fc2e800dc60 comp rx=0 tx=0).stop 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 shutdown_connections 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc2f011cd30 0x7fc2f01160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2f010aa90 0x7fc2f011c000 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800d950 tx=0x7fc2e800dc60 comp rx=0 tx=0).stop 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2d80b2940 0x7fc2d80b4e00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 --2- 172.21.15.47:0/572595387 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc2f0111d30 0x7fc2f011c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.730+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 >> 172.21.15.47:0/572595387 conn(0x7fc2f006e6c0 msgr2=0x7fc2f011e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.731+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 shutdown_connections 2024-05-11T12:32:20.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:20.731+0000 7fc2f81cf700 1 -- 172.21.15.47:0/572595387 wait complete. 2024-05-11T12:32:20.734 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:32:21.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:21 smithi047 ceph-mon[30375]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 757 MiB/s wr, 1.32k op/s 2024-05-11T12:32:21.641 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:21 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/572595387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:32:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:21 smithi050 ceph-mon[34072]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 757 MiB/s wr, 1.32k op/s 2024-05-11T12:32:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:21 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/572595387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:32:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:21 smithi064 ceph-mon[34537]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 757 MiB/s wr, 1.32k op/s 2024-05-11T12:32:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:21 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/572595387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:32:22.972 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:32:23.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:23 smithi047 ceph-mon[30375]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 884 MiB/s wr, 1.57k op/s 2024-05-11T12:32:23.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:23 smithi064 ceph-mon[34537]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 884 MiB/s wr, 1.57k op/s 2024-05-11T12:32:23.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:23 smithi050 ceph-mon[34072]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 884 MiB/s wr, 1.57k op/s 2024-05-11T12:32:23.972 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:32:25.686 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:25 smithi047 ceph-mon[30375]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 810 MiB/s wr, 1.42k op/s 2024-05-11T12:32:25.810 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:25 smithi050 ceph-mon[34072]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 810 MiB/s wr, 1.42k op/s 2024-05-11T12:32:25.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:25 smithi064 ceph-mon[34537]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 810 MiB/s wr, 1.42k op/s 2024-05-11T12:32:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:27 smithi050 ceph-mon[34072]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 715 MiB/s wr, 1.23k op/s 2024-05-11T12:32:28.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:27 smithi047 ceph-mon[30375]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 715 MiB/s wr, 1.23k op/s 2024-05-11T12:32:28.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:27 smithi064 ceph-mon[34537]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 715 MiB/s wr, 1.23k op/s 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.443+0000 7ff0d54a1700 1 -- 172.21.15.47:0/3720942049 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00b810 con 0x7ff0d010aa90 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.445+0000 7ff0d8707700 1 -- 172.21.15.47:0/3720942049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d010aa90 msgr2=0x7ff0d010af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.445+0000 7ff0d8707700 1 --2- 172.21.15.47:0/3720942049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d010aa90 0x7ff0d010af10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0098f0 tx=0x7ff0cc009c00 comp rx=0 tx=0).stop 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.445+0000 7ff0d8707700 1 -- 172.21.15.47:0/3720942049 shutdown_connections 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.445+0000 7ff0d8707700 1 --2- 172.21.15.47:0/3720942049 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d011cd30 0x7ff0d011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.445+0000 7ff0d8707700 1 --2- 172.21.15.47:0/3720942049 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0d0111d30 0x7ff0d010a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.445+0000 7ff0d8707700 1 --2- 172.21.15.47:0/3720942049 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d010aa90 0x7ff0d010af10 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0098f0 tx=0x7ff0cc009c00 comp rx=0 tx=0).stop 2024-05-11T12:32:29.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.445+0000 7ff0d8707700 1 -- 172.21.15.47:0/3720942049 >> 172.21.15.47:0/3720942049 conn(0x7ff0d006e6c0 msgr2=0x7ff0d006ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:29.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.448+0000 7ff0d8707700 1 -- 172.21.15.47:0/3720942049 shutdown_connections 2024-05-11T12:32:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.453+0000 7ff0d8707700 1 -- 172.21.15.47:0/3720942049 wait complete. 2024-05-11T12:32:29.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.454+0000 7ff0d8707700 1 Processor -- start 2024-05-11T12:32:29.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.454+0000 7ff0d8707700 1 -- start start 2024-05-11T12:32:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.455+0000 7ff0d8707700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0111d30 0x7ff0d01a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.455+0000 7ff0d8707700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0d011cd30 0x7ff0d01a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.455+0000 7ff0d8707700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d01ad740 0x7ff0d01adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.455+0000 7ff0d8707700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d01226f0 con 0x7ff0d0111d30 2024-05-11T12:32:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.455+0000 7ff0d8707700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0122570 con 0x7ff0d01ad740 2024-05-11T12:32:29.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.455+0000 7ff0d8707700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0122870 con 0x7ff0d011cd30 2024-05-11T12:32:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.458+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d01ad740 0x7ff0d01adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.458+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d01ad740 0x7ff0d01adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:34714/0 (socket says 172.21.15.47:34714) 2024-05-11T12:32:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.458+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/2775582823 learned_addr learned my addr 172.21.15.47:0/2775582823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.458+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0d011cd30 0x7ff0d01a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.461+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0111d30 0x7ff0d01a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.467+0000 7ff0d64a3700 1 -- 172.21.15.47:0/2775582823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0d011cd30 msgr2=0x7ff0d01a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.468+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0d011cd30 0x7ff0d01a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.468+0000 7ff0d64a3700 1 -- 172.21.15.47:0/2775582823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d01ad740 msgr2=0x7ff0d01adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.468+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d01ad740 0x7ff0d01adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.468+0000 7ff0d64a3700 1 -- 172.21.15.47:0/2775582823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01ae260 con 0x7ff0d0111d30 2024-05-11T12:32:29.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.469+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0111d30 0x7ff0d01a8e70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0006fd0 tx=0x7ff0c000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:29.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.470+0000 7ff0c77fe700 1 -- 172.21.15.47:0/2775582823 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0015020 con 0x7ff0d0111d30 2024-05-11T12:32:29.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.470+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01ae550 con 0x7ff0d0111d30 2024-05-11T12:32:29.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.470+0000 7ff0c77fe700 1 -- 172.21.15.47:0/2775582823 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c000be30 con 0x7ff0d0111d30 2024-05-11T12:32:29.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.470+0000 7ff0c77fe700 1 -- 172.21.15.47:0/2775582823 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0015020 con 0x7ff0d0111d30 2024-05-11T12:32:29.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.471+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01b5d80 con 0x7ff0d0111d30 2024-05-11T12:32:29.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.479+0000 7ff0e2f9db80 1 Processor -- start 2024-05-11T12:32:29.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.479+0000 7ff0c77fe700 1 -- 172.21.15.47:0/2775582823 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff0c0018020 con 0x7ff0d0111d30 2024-05-11T12:32:29.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.479+0000 7ff0e2f9db80 1 -- start start 2024-05-11T12:32:29.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.480+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9cd80 0x564a38d9d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.480+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 0x564a38d98510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.480+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d98f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.480+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564a38cea940 con 0x564a38d9d6a0 2024-05-11T12:32:29.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.480+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564a38c4fda0 con 0x564a38d98b30 2024-05-11T12:32:29.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.480+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a38ce6900 con 0x564a38d9cd80 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.483+0000 7ff0c77fe700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0bc0b2c70 0x7ff0bc0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.483+0000 7ff0d5ca2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d98f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.483+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9cd80 0x564a38d9d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.484+0000 7ff0c77fe700 1 -- 172.21.15.47:0/2775582823 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff0c0011630 con 0x7ff0d0111d30 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.485+0000 7ff0d5ca2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d98f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:34724/0 (socket says 172.21.15.47:34724) 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.485+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/979742366 learned_addr learned my addr 172.21.15.47:0/979742366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.485+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0bc0b2c70 0x7ff0bc0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.492+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/979742366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9cd80 msgr2=0x564a38d9d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.492+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/979742366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9cd80 0x564a38d9d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.492+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/979742366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 msgr2=0x564a38d98510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.492+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/979742366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 0x564a38d98510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.492+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/979742366 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a38c378d0 con 0x564a38d98b30 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.493+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0bc0b2c70 0x7ff0bc0b5130 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80094a0 tx=0x7ff0c8009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:29.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.493+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/979742366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d98f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009230 tx=0x7ff0cc009460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b540af167e51bfe server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.497+0000 7ff0c4ff9700 1 -- 172.21.15.47:0/979742366 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc005680 con 0x564a38d98b30 2024-05-11T12:32:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.497+0000 7ff0c4ff9700 1 -- 172.21.15.47:0/979742366 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0cc0094d0 con 0x564a38d98b30 2024-05-11T12:32:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/979742366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 msgr2=0x564a38d98f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/979742366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d98f90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009230 tx=0x7ff0cc009460 comp rx=0 tx=0).stop 2024-05-11T12:32:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/979742366 shutdown_connections 2024-05-11T12:32:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/979742366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d98f90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/979742366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9cd80 0x564a38d9d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/979742366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 0x564a38d98510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/979742366 >> 172.21.15.47:0/979742366 conn(0x564a38cc4b70 msgr2=0x564a38d92fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.498+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/979742366 shutdown_connections 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.499+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/979742366 wait complete. 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.499+0000 7ff0e2f9db80 1 -- start start 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.499+0000 7ff0e2f9db80 1 Processor -- start 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.500+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 0x564a38da4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.500+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9cd80 0x564a38da17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.500+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d9d6a0 0x564a38da1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.500+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564a38da51b0 con 0x564a38d9cd80 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.500+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564a38ca3ed0 con 0x564a38d9d6a0 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.500+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a38da5330 con 0x564a38d98b30 2024-05-11T12:32:29.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.500+0000 7ff0e2f9db80 10 client.0 _ll_register_callbacks cb 0x564a38c82950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.506+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d9d6a0 0x564a38da1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.506+0000 7ff0d64a3700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 0x564a38da4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.506+0000 7ff0d5ca2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9cd80 0x564a38da17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.506+0000 7ff0d5ca2700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9cd80 0x564a38da17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50394/0 (socket says 172.21.15.47:50394) 2024-05-11T12:32:29.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.506+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/1382867627 learned_addr learned my addr 172.21.15.47:0/1382867627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.507+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/1382867627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 msgr2=0x564a38da4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.507+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 0x564a38da4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.507+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/1382867627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d9d6a0 msgr2=0x564a38da1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.507+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d9d6a0 0x564a38da1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.507+0000 7ff0d5ca2700 1 -- 172.21.15.47:0/1382867627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a38ca1760 con 0x564a38d9cd80 2024-05-11T12:32:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.509+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9cd80 0x564a38da17f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc005e90 tx=0x7ff0cc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa27a15f7734894 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:29.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.509+0000 7ff0b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:32:29.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.511+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/1382867627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc031060 con 0x564a38d9cd80 2024-05-11T12:32:29.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.511+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0cc0094d0 con 0x564a38d9cd80 2024-05-11T12:32:29.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.511+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/1382867627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff0cc003680 con 0x564a38d9cd80 2024-05-11T12:32:29.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.512+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/1382867627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff0cc003880 con 0x564a38d9cd80 2024-05-11T12:32:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.518+0000 7ff0e2f9db80 10 client.15744 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:29.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.518+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564a38dac9a0 con 0x564a38d9cd80 2024-05-11T12:32:29.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.520+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/1382867627 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0cc0094d0 con 0x564a38d9cd80 2024-05-11T12:32:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.523+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/1382867627 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff0cc00faf0 con 0x564a38d9cd80 2024-05-11T12:32:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.523+0000 7ff0e2f9db80 10 client.15744 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.523+0000 7ff0e2f9db80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:29.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.523+0000 7ff0e2f9db80 20 client.15744 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.523+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38daef80 0x564a38db13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.523+0000 7ff0e2f9db80 4 client.15744 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:32:29.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.523+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a38cf8320 con 0x564a38daef80 2024-05-11T12:32:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.529+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38daef80 0x564a38db13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.535+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38daef80 0x564a38db13d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f6786a0edd3ac2e server_cookie=cabf912e22ade1c1 in_seq=0 out_seq=0 2024-05-11T12:32:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.535+0000 7ff0b9ffb700 10 client.15744 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:29.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.538+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/1382867627 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a38cf8320 con 0x564a38daef80 2024-05-11T12:32:29.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.538+0000 7ff0b9ffb700 10 client.15744 handle_command_reply: tid=0 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.539+0000 7ff0e2f9db80 1 client.15744 shutdown 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9cd80 msgr2=0x564a38da17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9cd80 0x564a38da17f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc005e90 tx=0x7ff0cc00b8f0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 shutdown_connections 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38daef80 0x564a38db13d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d9d6a0 0x564a38da1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 0x564a38da4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/1382867627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9cd80 0x564a38da17f0 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc005e90 tx=0x7ff0cc00b8f0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 >> 172.21.15.47:0/1382867627 conn(0x564a38cc4b70 msgr2=0x564a38d9e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.545+0000 7ff0b9ffb700 0 client.15744 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.546+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 shutdown_connections 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.546+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/1382867627 wait complete. 2024-05-11T12:32:29.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.547+0000 7ff0e2f9db80 20 client.15744 trim_cache size 0 max 16384 2024-05-11T12:32:29.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.631+0000 7ff0e2f9db80 1 Processor -- start 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.633+0000 7ff0e2f9db80 1 -- start start 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.633+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d8d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.633+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9d6a0 0x564a38db8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.633+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38db91f0 0x564a38de0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.633+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0117970 con 0x564a38db91f0 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.633+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d011a140 con 0x564a38d98b30 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.633+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0117f30 con 0x564a38d9d6a0 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.634+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38db91f0 0x564a38de0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.634+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38db91f0 0x564a38de0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50402/0 (socket says 172.21.15.47:50402) 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.634+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3184873858 learned_addr learned my addr 172.21.15.47:0/3184873858 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:29.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.635+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9d6a0 0x564a38db8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.635+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3184873858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9d6a0 msgr2=0x564a38db8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.635+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9d6a0 0x564a38db8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.635+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3184873858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 msgr2=0x564a38d8d940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.635+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d8d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.635+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3184873858 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a38dab630 con 0x564a38db91f0 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.636+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38db91f0 0x564a38de0700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8019b50 tx=0x7ff0c800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8b0d96d227eb42 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.639+0000 7ff0bbfff700 1 -- 172.21.15.47:0/3184873858 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8004bb0 con 0x564a38db91f0 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.640+0000 7ff0bbfff700 1 -- 172.21.15.47:0/3184873858 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c801bd90 con 0x564a38db91f0 2024-05-11T12:32:29.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.640+0000 7ff0bbfff700 1 -- 172.21.15.47:0/3184873858 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80267f0 con 0x564a38db91f0 2024-05-11T12:32:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.643+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3184873858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38db91f0 msgr2=0x564a38de0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.643+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38db91f0 0x564a38de0700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8019b50 tx=0x7ff0c800f000 comp rx=0 tx=0).stop 2024-05-11T12:32:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.644+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3184873858 shutdown_connections 2024-05-11T12:32:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.644+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38db91f0 0x564a38de0700 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.644+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38d98b30 0x564a38d8d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.644+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3184873858 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d9d6a0 0x564a38db8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.644+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3184873858 >> 172.21.15.47:0/3184873858 conn(0x564a38cc4b70 msgr2=0x564a38da0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:29.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.649+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3184873858 shutdown_connections 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.652+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3184873858 wait complete. 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.653+0000 7ff0e2f9db80 1 -- start start 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.653+0000 7ff0e2f9db80 1 Processor -- start 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.655+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 0x564a38daa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.655+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 0x564a38daab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.655+0000 7ff0e2f9db80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38db91f0 0x564a38dab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.655+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0118600 con 0x564a38d9d6a0 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.655+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0119a70 con 0x564a38db91f0 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.655+0000 7ff0e2f9db80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0118cd0 con 0x564a38d98b30 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.655+0000 7ff0e2f9db80 10 client.0 _ll_register_callbacks cb 0x564a38c82950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.656+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38db91f0 0x564a38dab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.656+0000 7ff0d6ca4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38db91f0 0x564a38dab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:34740/0 (socket says 172.21.15.47:34740) 2024-05-11T12:32:29.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.656+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3440713799 learned_addr learned my addr 172.21.15.47:0/3440713799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:29.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.658+0000 7ff0d5ca2700 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 0x564a38daab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.659+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3440713799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 msgr2=0x564a38daa640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:32:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.659+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 0x564a38daa640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.659+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3440713799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 msgr2=0x564a38daab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:29.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.659+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 0x564a38daab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.659+0000 7ff0d6ca4700 1 -- 172.21.15.47:0/3440713799 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a38d9ed20 con 0x564a38db91f0 2024-05-11T12:32:29.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.661+0000 7ff0d6ca4700 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38db91f0 0x564a38dab0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8026570 tx=0x7ff0c8004ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1cc9fa11ad84b1b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:29.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.661+0000 7ff0b9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:32:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.663+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/3440713799 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8034020 con 0x564a38db91f0 2024-05-11T12:32:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.663+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c801bd90 con 0x564a38db91f0 2024-05-11T12:32:29.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.663+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/3440713799 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff0c8026880 con 0x564a38db91f0 2024-05-11T12:32:29.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.664+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/3440713799 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff0c803f050 con 0x564a38db91f0 2024-05-11T12:32:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.666+0000 7ff0e2f9db80 10 client.25328 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:29.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.666+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564a38d95b50 con 0x564a38db91f0 2024-05-11T12:32:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.673+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/3440713799 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c801bd90 con 0x564a38db91f0 2024-05-11T12:32:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.675+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/3440713799 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff0c8015040 con 0x564a38db91f0 2024-05-11T12:32:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.675+0000 7ff0e2f9db80 10 client.25328 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.675+0000 7ff0e2f9db80 10 client.25328 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:29.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.675+0000 7ff0e2f9db80 20 client.25328 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.675+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38e3f290 0x564a38e41680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.675+0000 7ff0e2f9db80 4 client.25328 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:32:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.675+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a38cf23c0 con 0x564a38e3f290 2024-05-11T12:32:29.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.677+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38e3f290 0x564a38e41680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.679+0000 7ff0d64a3700 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38e3f290 0x564a38e41680 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92a940fd015c7c0 server_cookie=d697f4c64a31c17d in_seq=0 out_seq=0 2024-05-11T12:32:29.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.679+0000 7ff0b9ffb700 10 client.25328 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:29.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.681+0000 7ff0b9ffb700 1 -- 172.21.15.47:0/3440713799 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a38cf23c0 con 0x564a38e3f290 2024-05-11T12:32:29.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.681+0000 7ff0b9ffb700 10 client.25328 handle_command_reply: tid=0 2024-05-11T12:32:29.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.682+0000 7ff0e2f9db80 1 client.25328 shutdown 2024-05-11T12:32:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.693+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38db91f0 msgr2=0x564a38dab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.693+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38db91f0 0x564a38dab0c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8026570 tx=0x7ff0c8004ca0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.695+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 shutdown_connections 2024-05-11T12:32:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.695+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a38e3f290 0x564a38e41680 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.695+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a38db91f0 0x564a38dab0c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.695+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a38d98b30 0x564a38daa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.695+0000 7ff0e2f9db80 1 --2- 172.21.15.47:0/3440713799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a38d9d6a0 0x564a38daab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.695+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 >> 172.21.15.47:0/3440713799 conn(0x564a38cc4b70 msgr2=0x564a38da10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:29.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.695+0000 7ff0b9ffb700 0 client.25328 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:29.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.699+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 shutdown_connections 2024-05-11T12:32:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.700+0000 7ff0e2f9db80 1 -- 172.21.15.47:0/3440713799 wait complete. 2024-05-11T12:32:29.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.700+0000 7ff0e2f9db80 20 client.25328 trim_cache size 0 max 16384 2024-05-11T12:32:29.701 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:32:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.705+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0bc0b2c70 msgr2=0x7ff0bc0b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.705+0000 7ff0d8707700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0bc0b2c70 0x7ff0bc0b5130 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80094a0 tx=0x7ff0c8009510 comp rx=0 tx=0).stop 2024-05-11T12:32:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0111d30 msgr2=0x7ff0d01a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0111d30 0x7ff0d01a8e70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0006fd0 tx=0x7ff0c000b730 comp rx=0 tx=0).stop 2024-05-11T12:32:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 shutdown_connections 2024-05-11T12:32:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff0bc0b2c70 0x7ff0bc0b5130 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0d01ad740 0x7ff0d01adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0d0111d30 0x7ff0d01a8e70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 --2- 172.21.15.47:0/2775582823 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff0d011cd30 0x7ff0d01a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 >> 172.21.15.47:0/2775582823 conn(0x7ff0d006e6c0 msgr2=0x7ff0d011ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 shutdown_connections 2024-05-11T12:32:29.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:29.706+0000 7ff0d8707700 1 -- 172.21.15.47:0/2775582823 wait complete. 2024-05-11T12:32:30.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:29 smithi047 ceph-mon[30375]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 1.53k op/s 2024-05-11T12:32:30.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:29 smithi064 ceph-mon[34537]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 1.53k op/s 2024-05-11T12:32:30.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:29 smithi050 ceph-mon[34072]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 915 MiB/s wr, 1.53k op/s 2024-05-11T12:32:32.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:31 smithi047 ceph-mon[30375]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 724 MiB/s wr, 1.21k op/s 2024-05-11T12:32:32.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:31 smithi064 ceph-mon[34537]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 724 MiB/s wr, 1.21k op/s 2024-05-11T12:32:32.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:31 smithi050 ceph-mon[34072]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 724 MiB/s wr, 1.21k op/s 2024-05-11T12:32:32.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:32:32.884 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:32:33.829 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:32:34.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:33 smithi047 ceph-mon[30375]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 872 MiB/s wr, 1.44k op/s 2024-05-11T12:32:34.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:33 smithi064 ceph-mon[34537]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 872 MiB/s wr, 1.44k op/s 2024-05-11T12:32:34.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:33 smithi050 ceph-mon[34072]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 872 MiB/s wr, 1.44k op/s 2024-05-11T12:32:36.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:35 smithi047 ceph-mon[30375]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 798 MiB/s wr, 1.30k op/s 2024-05-11T12:32:36.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:35 smithi064 ceph-mon[34537]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 798 MiB/s wr, 1.30k op/s 2024-05-11T12:32:36.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:35 smithi050 ceph-mon[34072]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 798 MiB/s wr, 1.30k op/s 2024-05-11T12:32:37.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:36 smithi047 ceph-mon[30375]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 705 MiB/s wr, 1.16k op/s 2024-05-11T12:32:37.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:36 smithi064 ceph-mon[34537]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 705 MiB/s wr, 1.16k op/s 2024-05-11T12:32:37.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:36 smithi050 ceph-mon[34072]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 705 MiB/s wr, 1.16k op/s 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.515+0000 7f956f6b9700 1 -- 172.21.15.47:0/3808756680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568103440 msgr2=0x7f95681070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.515+0000 7f956f6b9700 1 --2- 172.21.15.47:0/3808756680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568103440 0x7f95681070b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f956401a9d0 tx=0x7f956401ace0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.516+0000 7f956f6b9700 1 -- 172.21.15.47:0/3808756680 shutdown_connections 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.516+0000 7f956f6b9700 1 --2- 172.21.15.47:0/3808756680 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f95681077e0 0x7f956810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.516+0000 7f956f6b9700 1 --2- 172.21.15.47:0/3808756680 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f9568102a90 0x7f9568102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.516+0000 7f956f6b9700 1 --2- 172.21.15.47:0/3808756680 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568103440 0x7f95681070b0 secure :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f956401a9d0 tx=0x7f956401ace0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.516+0000 7f956f6b9700 1 -- 172.21.15.47:0/3808756680 >> 172.21.15.47:0/3808756680 conn(0x7f9568079400 msgr2=0x7f9568079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:39.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.519+0000 7f956f6b9700 1 -- 172.21.15.47:0/3808756680 shutdown_connections 2024-05-11T12:32:39.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:39 smithi047 ceph-mon[30375]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 949 MiB/s wr, 1.59k op/s 2024-05-11T12:32:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.519+0000 7f956f6b9700 1 -- 172.21.15.47:0/3808756680 wait complete. 2024-05-11T12:32:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.521+0000 7f956f6b9700 1 Processor -- start 2024-05-11T12:32:39.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.522+0000 7f956f6b9700 1 -- start start 2024-05-11T12:32:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.523+0000 7f956f6b9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9568102a90 0x7f956810ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.523+0000 7f956f6b9700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95681077e0 0x7f9568109e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.523+0000 7f956f6b9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f956810a3d0 0x7f956810a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.523+0000 7f956f6b9700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f956806f830 con 0x7f95681077e0 2024-05-11T12:32:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.523+0000 7f956f6b9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f956806f6b0 con 0x7f9568102a90 2024-05-11T12:32:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.523+0000 7f956f6b9700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f956806f9b0 con 0x7f956810a3d0 2024-05-11T12:32:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.524+0000 7f956cc54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95681077e0 0x7f9568109e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.524+0000 7f956cc54700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95681077e0 0x7f9568109e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56070/0 (socket says 172.21.15.47:56070) 2024-05-11T12:32:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.525+0000 7f956cc54700 1 -- 172.21.15.47:0/42531425 learned_addr learned my addr 172.21.15.47:0/42531425 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.526+0000 7f956cc54700 1 -- 172.21.15.47:0/42531425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f956810a3d0 msgr2=0x7f956810a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.526+0000 7f956cc54700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f956810a3d0 0x7f956810a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.526+0000 7f956cc54700 1 -- 172.21.15.47:0/42531425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9568102a90 msgr2=0x7f956810ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.526+0000 7f956cc54700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9568102a90 0x7f956810ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.526+0000 7f956cc54700 1 -- 172.21.15.47:0/42531425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956819bdd0 con 0x7f95681077e0 2024-05-11T12:32:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.527+0000 7f956cc54700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95681077e0 0x7f9568109e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f956401c590 tx=0x7f956401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:39.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.528+0000 7f955e7fc700 1 -- 172.21.15.47:0/42531425 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640036f0 con 0x7f95681077e0 2024-05-11T12:32:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.528+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956819c060 con 0x7f95681077e0 2024-05-11T12:32:39.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.528+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956819c490 con 0x7f95681077e0 2024-05-11T12:32:39.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.538+0000 7f955e7fc700 1 -- 172.21.15.47:0/42531425 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9564003da0 con 0x7f95681077e0 2024-05-11T12:32:39.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.539+0000 7f955e7fc700 1 -- 172.21.15.47:0/42531425 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9564028d30 con 0x7f95681077e0 2024-05-11T12:32:39.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.546+0000 7f9579f4fb80 1 Processor -- start 2024-05-11T12:32:39.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.551+0000 7f955e7fc700 1 -- 172.21.15.47:0/42531425 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f956403b440 con 0x7f95681077e0 2024-05-11T12:32:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.553+0000 7f9579f4fb80 1 -- start start 2024-05-11T12:32:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.556+0000 7f955e7fc700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f95540b2a90 0x7f95540b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.560+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ee71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.560+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 0x55e9e5ee2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.560+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ee3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.560+0000 7f9579f4fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e9e5e349c0 con 0x55e9e5ee7720 2024-05-11T12:32:39.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.560+0000 7f9579f4fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e9e5d99e20 con 0x55e9e5ee6e00 2024-05-11T12:32:39.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.560+0000 7f9579f4fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9e5e30980 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.561+0000 7f956d455700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ee71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.558+0000 7f956dc56700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f95540b2a90 0x7f95540b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.562+0000 7f955e7fc700 1 -- 172.21.15.47:0/42531425 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f95640f42e0 con 0x7f95681077e0 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.562+0000 7f956dc56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ee3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.562+0000 7f956dc56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ee3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:41188/0 (socket says 172.21.15.47:41188) 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.562+0000 7f956dc56700 1 -- 172.21.15.47:0/1506767721 learned_addr learned my addr 172.21.15.47:0/1506767721 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.563+0000 7f956dc56700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f95540b2a90 0x7f95540b4f50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9560010640 tx=0x7f9560010a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.563+0000 7f956dc56700 1 -- 172.21.15.47:0/1506767721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 msgr2=0x55e9e5ee71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.563+0000 7f956dc56700 1 --2- 172.21.15.47:0/1506767721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ee71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.563+0000 7f956dc56700 1 -- 172.21.15.47:0/1506767721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 msgr2=0x55e9e5ee2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.563+0000 7f956dc56700 1 --2- 172.21.15.47:0/1506767721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 0x55e9e5ee2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.563+0000 7f956dc56700 1 -- 172.21.15.47:0/1506767721 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e5d81950 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.564+0000 7f956dc56700 1 --2- 172.21.15.47:0/1506767721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ee3010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f956000e770 tx=0x7f9560018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1127386e0801c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.569+0000 7f95537fe700 1 -- 172.21.15.47:0/1506767721 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560006e60 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.569+0000 7f95537fe700 1 -- 172.21.15.47:0/1506767721 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956001cad0 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.569+0000 7f95537fe700 1 -- 172.21.15.47:0/1506767721 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956000ba20 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.571+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1506767721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee2bb0 msgr2=0x55e9e5ee3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.571+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1506767721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ee3010 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f956000e770 tx=0x7f9560018000 comp rx=0 tx=0).stop 2024-05-11T12:32:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.571+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1506767721 shutdown_connections 2024-05-11T12:32:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.571+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1506767721 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ee3010 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.571+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1506767721 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ee71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.571+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1506767721 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 0x55e9e5ee2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.571+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1506767721 >> 172.21.15.47:0/1506767721 conn(0x55e9e5e0ebf0 msgr2=0x55e9e5edd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.574+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1506767721 shutdown_connections 2024-05-11T12:32:39.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.574+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1506767721 wait complete. 2024-05-11T12:32:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.574+0000 7f9579f4fb80 1 -- start start 2024-05-11T12:32:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.575+0000 7f9579f4fb80 1 Processor -- start 2024-05-11T12:32:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.576+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5eee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.576+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 0x55e9e5eeba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.576+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5eebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.576+0000 7f9579f4fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55e9e5eef290 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.576+0000 7f9579f4fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55e9e5dedf50 con 0x55e9e5ee6e00 2024-05-11T12:32:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.576+0000 7f9579f4fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55e9e5eef410 con 0x55e9e5ee7720 2024-05-11T12:32:39.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.576+0000 7f9579f4fb80 10 client.0 _ll_register_callbacks cb 0x55e9e5dcc9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:39.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.577+0000 7f956dc56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5eebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.577+0000 7f956dc56700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5eebf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:41192/0 (socket says 172.21.15.47:41192) 2024-05-11T12:32:39.581 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.577+0000 7f956dc56700 1 -- 172.21.15.47:0/1547920020 learned_addr learned my addr 172.21.15.47:0/1547920020 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:39.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.582+0000 7f956d455700 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5eee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.582+0000 7f956dc56700 1 -- 172.21.15.47:0/1547920020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 msgr2=0x55e9e5eeba00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:32:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.582+0000 7f956dc56700 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 0x55e9e5eeba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.582+0000 7f956dc56700 1 -- 172.21.15.47:0/1547920020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee2bb0 msgr2=0x55e9e5eee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.582+0000 7f956dc56700 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5eee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.582+0000 7f956dc56700 1 -- 172.21.15.47:0/1547920020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e5deb7e0 con 0x55e9e5ee7720 2024-05-11T12:32:39.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.583+0000 7f956dc56700 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5eebf40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f956001ca50 tx=0x7f956000a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d385a1a8865d5daa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.583+0000 7f9550ff9700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:32:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.583+0000 7f9550ff9700 1 -- 172.21.15.47:0/1547920020 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956002b070 con 0x55e9e5ee7720 2024-05-11T12:32:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.583+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f956001cad0 con 0x55e9e5ee7720 2024-05-11T12:32:39.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.583+0000 7f9550ff9700 1 -- 172.21.15.47:0/1547920020 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9560003ca0 con 0x55e9e5ee7720 2024-05-11T12:32:39.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.584+0000 7f9550ff9700 1 -- 172.21.15.47:0/1547920020 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f956000b620 con 0x55e9e5ee7720 2024-05-11T12:32:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.585+0000 7f9579f4fb80 10 client.25363 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.585+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e9e5ef9a40 con 0x55e9e5ee7720 2024-05-11T12:32:39.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.586+0000 7f9550ff9700 1 -- 172.21.15.47:0/1547920020 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f956001cad0 con 0x55e9e5ee7720 2024-05-11T12:32:39.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.587+0000 7f9550ff9700 1 -- 172.21.15.47:0/1547920020 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f956002f840 con 0x55e9e5ee7720 2024-05-11T12:32:39.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.587+0000 7f9579f4fb80 10 client.25363 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:39.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.587+0000 7f9579f4fb80 10 client.25363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:39.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.587+0000 7f9579f4fb80 20 client.25363 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:39.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.587+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5efc3c0 0x55e9e5efe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.587+0000 7f9579f4fb80 4 client.25363 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:32:39.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.587+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9e5e423a0 con 0x55e9e5efc3c0 2024-05-11T12:32:39.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.588+0000 7f956d455700 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5efc3c0 0x55e9e5efe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.591+0000 7f956d455700 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5efc3c0 0x55e9e5efe7e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9118d1e62497bedc server_cookie=eee5a11a0fd4a6b3 in_seq=0 out_seq=0 2024-05-11T12:32:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.591+0000 7f9550ff9700 10 client.25363 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.595+0000 7f9550ff9700 1 -- 172.21.15.47:0/1547920020 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e9e5e423a0 con 0x55e9e5efc3c0 2024-05-11T12:32:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.595+0000 7f9550ff9700 10 client.25363 handle_command_reply: tid=0 2024-05-11T12:32:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.595+0000 7f9579f4fb80 1 client.25363 shutdown 2024-05-11T12:32:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.596+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 msgr2=0x55e9e5eebf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.596+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5eebf40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f956001ca50 tx=0x7f956000a610 comp rx=0 tx=0).stop 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.597+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 shutdown_connections 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.597+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5efc3c0 0x55e9e5efe7e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.597+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5eebf40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.597+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5eee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.597+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1547920020 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee6e00 0x55e9e5eeba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.597+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 >> 172.21.15.47:0/1547920020 conn(0x55e9e5e0ebf0 msgr2=0x55e9e5ee8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.597+0000 7f9550ff9700 0 client.25363 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.598+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 shutdown_connections 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.598+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1547920020 wait complete. 2024-05-11T12:32:39.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.598+0000 7f9579f4fb80 20 client.25363 trim_cache size 0 max 16384 2024-05-11T12:32:39.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:39 smithi064 ceph-mon[34537]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 949 MiB/s wr, 1.59k op/s 2024-05-11T12:32:39.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.644+0000 7f9579f4fb80 1 Processor -- start 2024-05-11T12:32:39.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.645+0000 7f9579f4fb80 1 -- start start 2024-05-11T12:32:39.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.645+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ed79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5f040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f9579f4fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f954c00b710 con 0x55e9e5ee6e00 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f956d455700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f956d455700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:39240/0 (socket says 172.21.15.47:39240) 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f956d455700 1 -- 172.21.15.47:0/2384055705 learned_addr learned my addr 172.21.15.47:0/2384055705 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f9579f4fb80 1 -- 172.21.15.47:0/2384055705 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f954c00dee0 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.646+0000 7f9579f4fb80 1 -- 172.21.15.47:0/2384055705 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f954c00bcd0 con 0x55e9e5ee7720 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.647+0000 7f956cc54700 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ed79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.647+0000 7f956dc56700 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5f040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.648+0000 7f956cc54700 1 -- 172.21.15.47:0/2384055705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 msgr2=0x55e9e5f040b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.648+0000 7f956cc54700 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5f040b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.648+0000 7f956cc54700 1 -- 172.21.15.47:0/2384055705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 msgr2=0x55e9e5ef83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.648+0000 7f956cc54700 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.648+0000 7f956cc54700 1 -- 172.21.15.47:0/2384055705 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9e5ee9f70 con 0x55e9e5ee6e00 2024-05-11T12:32:39.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.648+0000 7f956cc54700 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ed79c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55e9e5eecfd0 tx=0x7f956401a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35888eb95a82e855 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.650+0000 7f9552ffd700 1 -- 172.21.15.47:0/2384055705 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640f3070 con 0x55e9e5ee6e00 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.650+0000 7f9552ffd700 1 -- 172.21.15.47:0/2384055705 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95640bb370 con 0x55e9e5ee6e00 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.651+0000 7f9552ffd700 1 -- 172.21.15.47:0/2384055705 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95640bfc10 con 0x55e9e5ee6e00 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.652+0000 7f9579f4fb80 1 -- 172.21.15.47:0/2384055705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee6e00 msgr2=0x55e9e5ed79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.652+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ed79c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55e9e5eecfd0 tx=0x7f956401a6b0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.652+0000 7f9579f4fb80 1 -- 172.21.15.47:0/2384055705 shutdown_connections 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.652+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee7720 0x55e9e5f040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.652+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.652+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/2384055705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ed79c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.652+0000 7f9579f4fb80 1 -- 172.21.15.47:0/2384055705 >> 172.21.15.47:0/2384055705 conn(0x55e9e5e0ebf0 msgr2=0x55e9e5eed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.653+0000 7f9579f4fb80 1 -- 172.21.15.47:0/2384055705 shutdown_connections 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.653+0000 7f9579f4fb80 1 -- 172.21.15.47:0/2384055705 wait complete. 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.654+0000 7f9579f4fb80 1 -- start start 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.654+0000 7f9579f4fb80 1 Processor -- start 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.655+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.655+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ef4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.655+0000 7f9579f4fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 0x55e9e5ef0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.655+0000 7f956d455700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f956d455700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:39252/0 (socket says 172.21.15.47:39252) 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f956cc54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ef4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f956cc54700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ef4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:41214/0 (socket says 172.21.15.47:41214) 2024-05-11T12:32:39.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f956d455700 1 -- 172.21.15.47:0/1708910686 learned_addr learned my addr 172.21.15.47:0/1708910686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f954c00c3a0 con 0x55e9e5ee7720 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f954c00d810 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f954c00ca70 con 0x55e9e5ee6e00 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.656+0000 7f9579f4fb80 10 client.0 _ll_register_callbacks cb 0x55e9e5dcc9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.661+0000 7f956d455700 1 -- 172.21.15.47:0/1708910686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee6e00 msgr2=0x55e9e5ef4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.661+0000 7f956d455700 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ef4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.661+0000 7f956d455700 1 -- 172.21.15.47:0/1708910686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 msgr2=0x55e9e5ef0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.661+0000 7f956d455700 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 0x55e9e5ef0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.661+0000 7f956d455700 1 -- 172.21.15.47:0/1708910686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9e5ef1680 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.662+0000 7f956d455700 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef48b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f955800ae00 tx=0x7f955800ae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3da650b5f0f8d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.662+0000 7f9550ff9700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.662+0000 7f9550ff9700 1 -- 172.21.15.47:0/1708910686 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558028060 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.663+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95640bb370 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.663+0000 7f9550ff9700 1 -- 172.21.15.47:0/1708910686 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f9558018860 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.663+0000 7f9550ff9700 1 -- 172.21.15.47:0/1708910686 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9558018a60 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.663+0000 7f9579f4fb80 10 client.25334 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.664+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e9e5f67930 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.664+0000 7f9550ff9700 1 -- 172.21.15.47:0/1708910686 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95640bb370 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.665+0000 7f9550ff9700 1 -- 172.21.15.47:0/1708910686 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f95580144c0 con 0x55e9e5ee2bb0 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.665+0000 7f9579f4fb80 10 client.25334 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.665+0000 7f9579f4fb80 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.665+0000 7f9579f4fb80 20 client.25334 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.665+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5f68070 0x55e9e5f6a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.665+0000 7f9579f4fb80 4 client.25334 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.665+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e9e5e3c440 con 0x55e9e5f68070 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.666+0000 7f956cc54700 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5f68070 0x55e9e5f6a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.667+0000 7f9550ff9700 10 client.25334 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:39.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.667+0000 7f956cc54700 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5f68070 0x55e9e5f6a4d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26520fd291d1f4f4 server_cookie=eb179e780aeef9f8 in_seq=0 out_seq=0 2024-05-11T12:32:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.670+0000 7f9550ff9700 1 -- 172.21.15.47:0/1708910686 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e9e5e3c440 con 0x55e9e5f68070 2024-05-11T12:32:39.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.670+0000 7f9550ff9700 10 client.25334 handle_command_reply: tid=0 2024-05-11T12:32:39.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.671+0000 7f9579f4fb80 1 client.25334 shutdown 2024-05-11T12:32:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.671+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 msgr2=0x55e9e5ef48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.671+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef48b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f955800ae00 tx=0x7f955800ae30 comp rx=0 tx=0).stop 2024-05-11T12:32:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 shutdown_connections 2024-05-11T12:32:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55e9e5f68070 0x55e9e5f6a4d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55e9e5ee7720 0x55e9e5ef0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55e9e5ee2bb0 0x55e9e5ef48b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 --2- 172.21.15.47:0/1708910686 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55e9e5ee6e00 0x55e9e5ef4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 >> 172.21.15.47:0/1708910686 conn(0x55e9e5e0ebf0 msgr2=0x55e9e5eed890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9550ff9700 0 client.25334 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 shutdown_connections 2024-05-11T12:32:39.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.672+0000 7f9579f4fb80 1 -- 172.21.15.47:0/1708910686 wait complete. 2024-05-11T12:32:39.676 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:32:39.676 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:32:39.676 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "6c81c6b7-c89b-4d73-b473-44bc777db899", 2024-05-11T12:32:39.676 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:32:39.677 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:32:39.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.673+0000 7f9579f4fb80 20 client.25334 trim_cache size 0 max 16384 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f95540b2a90 msgr2=0x7f95540b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f95540b2a90 0x7f95540b4f50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9560010640 tx=0x7f9560010a00 comp rx=0 tx=0).stop 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95681077e0 msgr2=0x7f9568109e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95681077e0 0x7f9568109e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f956401c590 tx=0x7f956401cc70 comp rx=0 tx=0).stop 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 shutdown_connections 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f956810a3d0 0x7f956810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f95540b2a90 0x7f95540b4f50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9568102a90 0x7f956810ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 --2- 172.21.15.47:0/42531425 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f95681077e0 0x7f9568109e90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:39.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.676+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 >> 172.21.15.47:0/42531425 conn(0x7f9568079400 msgr2=0x7f9568106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:39.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.680+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 shutdown_connections 2024-05-11T12:32:39.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:39.680+0000 7f956f6b9700 1 -- 172.21.15.47:0/42531425 wait complete. 2024-05-11T12:32:39.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:39 smithi050 ceph-mon[34072]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 949 MiB/s wr, 1.59k op/s 2024-05-11T12:32:41.507 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:32:41.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:41 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:32:41.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:41 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:41.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:41 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:32:41.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:41 smithi047 ceph-mon[30375]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 752 MiB/s wr, 1.28k op/s 2024-05-11T12:32:41.685 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:41 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:41.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:41 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:32:41.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:41 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:41.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:41 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:32:41.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:41 smithi050 ceph-mon[34072]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 752 MiB/s wr, 1.28k op/s 2024-05-11T12:32:41.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:41 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:41 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:32:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:41 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:41.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:41 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:32:41.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:41 smithi064 ceph-mon[34537]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 752 MiB/s wr, 1.28k op/s 2024-05-11T12:32:41.865 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:41 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:32:42.533 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:32:43.525 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:43 smithi047 ceph-mon[30375]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 893 MiB/s wr, 1.53k op/s 2024-05-11T12:32:43.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:43 smithi050 ceph-mon[34072]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 893 MiB/s wr, 1.53k op/s 2024-05-11T12:32:43.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:43 smithi064 ceph-mon[34537]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 893 MiB/s wr, 1.53k op/s 2024-05-11T12:32:44.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:44 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:32:44.782 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:44 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:32:44.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:44 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:32:45.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:45 smithi047 ceph-mon[30375]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.42k op/s 2024-05-11T12:32:45.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:45 smithi064 ceph-mon[34537]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.42k op/s 2024-05-11T12:32:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:45 smithi050 ceph-mon[34072]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 816 MiB/s wr, 1.42k op/s 2024-05-11T12:32:47.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:47 smithi050 ceph-mon[34072]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 723 MiB/s wr, 1.27k op/s 2024-05-11T12:32:47.959 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:47 smithi047 ceph-mon[30375]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 723 MiB/s wr, 1.27k op/s 2024-05-11T12:32:48.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:47 smithi064 ceph-mon[34537]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 723 MiB/s wr, 1.27k op/s 2024-05-11T12:32:49.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:49 smithi050 ceph-mon[34072]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 965 MiB/s wr, 1.84k op/s 2024-05-11T12:32:49.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:49 smithi047 ceph-mon[30375]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 965 MiB/s wr, 1.84k op/s 2024-05-11T12:32:50.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:49 smithi064 ceph-mon[34537]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 965 MiB/s wr, 1.84k op/s 2024-05-11T12:32:50.515 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:32:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:32:50.515 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:32:50.515 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:32:50.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.269+0000 7f3955eb1700 1 -- 172.21.15.47:0/2606988061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3950102f90 msgr2=0x7f3950106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.269+0000 7f3955eb1700 1 --2- 172.21.15.47:0/2606988061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3950102f90 0x7f3950106fe0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3940009990 tx=0x7f3940009ca0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.270+0000 7f3955eb1700 1 -- 172.21.15.47:0/2606988061 shutdown_connections 2024-05-11T12:32:50.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.270+0000 7f3955eb1700 1 --2- 172.21.15.47:0/2606988061 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39501076c0 0x7f395010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.270+0000 7f3955eb1700 1 --2- 172.21.15.47:0/2606988061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501025e0 0x7f39501029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.270+0000 7f3955eb1700 1 --2- 172.21.15.47:0/2606988061 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3950102f90 0x7f3950106fe0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.270+0000 7f3955eb1700 1 -- 172.21.15.47:0/2606988061 >> 172.21.15.47:0/2606988061 conn(0x7f39500fde70 msgr2=0x7f3950100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.271+0000 7f3955eb1700 1 -- 172.21.15.47:0/2606988061 shutdown_connections 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.271+0000 7f3955eb1700 1 -- 172.21.15.47:0/2606988061 wait complete. 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 Processor -- start 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 -- start start 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39501025e0 0x7f3950117c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3950102f90 0x7f3950111cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501076c0 0x7f3950112200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3950110710 con 0x7f39501076c0 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f3950110590 con 0x7f3950102f90 2024-05-11T12:32:50.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.272+0000 7f3955eb1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f3950110890 con 0x7f39501025e0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.273+0000 7f39556b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501076c0 0x7f3950112200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.273+0000 7f39556b0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501076c0 0x7f3950112200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49990/0 (socket says 172.21.15.47:49990) 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.273+0000 7f39556b0700 1 -- 172.21.15.47:0/1076474888 learned_addr learned my addr 172.21.15.47:0/1076474888 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.274+0000 7f3954eaf700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39501025e0 0x7f3950117c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.274+0000 7f39556b0700 1 -- 172.21.15.47:0/1076474888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39501025e0 msgr2=0x7f3950117c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.274+0000 7f39556b0700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39501025e0 0x7f3950117c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.274+0000 7f39556b0700 1 -- 172.21.15.47:0/1076474888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3950102f90 msgr2=0x7f3950111cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.274+0000 7f39556b0700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3950102f90 0x7f3950111cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.274+0000 7f39556b0700 1 -- 172.21.15.47:0/1076474888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39501127f0 con 0x7f39501076c0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.275+0000 7f39556b0700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501076c0 0x7f3950112200 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f394800ada0 tx=0x7f394800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.275+0000 7f394dffb700 1 -- 172.21.15.47:0/1076474888 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948017020 con 0x7f39501076c0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.275+0000 7f394dffb700 1 -- 172.21.15.47:0/1076474888 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394800ce30 con 0x7f39501076c0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.275+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3950112ae0 con 0x7f39501076c0 2024-05-11T12:32:50.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.275+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3950073310 con 0x7f39501076c0 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.276+0000 7f394dffb700 1 -- 172.21.15.47:0/1076474888 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480054a0 con 0x7f39501076c0 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.294+0000 7f394dffb700 1 -- 172.21.15.47:0/1076474888 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3948004ad0 con 0x7f39501076c0 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.295+0000 7f39619a9b80 1 Processor -- start 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.296+0000 7f394dffb700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f39380b29c0 0x7f39380b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.296+0000 7f394dffb700 1 -- 172.21.15.47:0/1076474888 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f39480e1960 con 0x7f39501076c0 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.297+0000 7f39619a9b80 1 -- start start 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.297+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.297+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 0x556743354510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.297+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.297+0000 7f39619a9b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5567432a6940 con 0x55674335a920 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.297+0000 7f39619a9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55674320bda0 con 0x55674335a000 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.297+0000 7f39619a9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5567432a2900 con 0x556743354b30 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.298+0000 7f3954eaf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.298+0000 7f3954eaf700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:33412/0 (socket says 172.21.15.47:33412) 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.298+0000 7f3954eaf700 1 -- 172.21.15.47:0/1184354047 learned_addr learned my addr 172.21.15.47:0/1184354047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.298+0000 7f39556b0700 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.299+0000 7f394ffff700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f39380b29c0 0x7f39380b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.300+0000 7f3954eaf700 1 -- 172.21.15.47:0/1184354047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 msgr2=0x556743354f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.300+0000 7f3954eaf700 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.300+0000 7f3954eaf700 1 -- 172.21.15.47:0/1184354047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 msgr2=0x556743354510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.300+0000 7f3954eaf700 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 0x556743354510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.300+0000 7f3954eaf700 1 -- 172.21.15.47:0/1184354047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567431f38d0 con 0x55674335a000 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.301+0000 7f3954eaf700 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335a3e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3944008c60 tx=0x7f3944008f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45e1d800c507153 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.301+0000 7f394ffff700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f39380b29c0 0x7f39380b4e80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f394000d7d0 tx=0x7f3940005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.302+0000 7f3936ffd700 1 -- 172.21.15.47:0/1184354047 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944014070 con 0x55674335a000 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.302+0000 7f3936ffd700 1 -- 172.21.15.47:0/1184354047 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f394400ed70 con 0x55674335a000 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.302+0000 7f3936ffd700 1 -- 172.21.15.47:0/1184354047 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944010430 con 0x55674335a000 2024-05-11T12:32:50.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 -- 172.21.15.47:0/1184354047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 msgr2=0x55674335a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335a3e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3944008c60 tx=0x7f3944008f70 comp rx=0 tx=0).stop 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 -- 172.21.15.47:0/1184354047 shutdown_connections 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335a3e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 --2- 172.21.15.47:0/1184354047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 0x556743354510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 -- 172.21.15.47:0/1184354047 >> 172.21.15.47:0/1184354047 conn(0x556743280b70 msgr2=0x55674334f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 -- 172.21.15.47:0/1184354047 shutdown_connections 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.303+0000 7f39619a9b80 1 -- 172.21.15.47:0/1184354047 wait complete. 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.304+0000 7f39619a9b80 1 -- start start 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.304+0000 7f39619a9b80 1 Processor -- start 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.305+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743360600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.305+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.305+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x55674335df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.305+0000 7f39619a9b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x556743361440 con 0x556743354b30 2024-05-11T12:32:50.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.305+0000 7f39619a9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55674325fed0 con 0x55674335a000 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.305+0000 7f39619a9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5567433615c0 con 0x55674335a920 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.305+0000 7f39619a9b80 10 client.0 _ll_register_callbacks cb 0x55674323e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.306+0000 7f3954eaf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743360600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.306+0000 7f3954eaf700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743360600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50016/0 (socket says 172.21.15.47:50016) 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.306+0000 7f3954eaf700 1 -- 172.21.15.47:0/2025522230 learned_addr learned my addr 172.21.15.47:0/2025522230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.306+0000 7f39556b0700 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x55674335df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.307+0000 7f394ffff700 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.307+0000 7f39556b0700 1 -- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 msgr2=0x55674335d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.307+0000 7f39556b0700 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.307+0000 7f39556b0700 1 -- 172.21.15.47:0/2025522230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 msgr2=0x556743360600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.307+0000 7f39556b0700 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743360600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.307+0000 7f39556b0700 1 -- 172.21.15.47:0/2025522230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674325d760 con 0x55674335a920 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.311+0000 7f39556b0700 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x55674335df30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39480df300 tx=0x7f3948007ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed9b6d2b37ce13b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:50.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.311+0000 7f392ffff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.312+0000 7f392ffff700 1 -- 172.21.15.47:0/2025522230 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480a8ee0 con 0x55674335a920 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.312+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394400ed70 con 0x55674335a920 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.312+0000 7f392ffff700 1 -- 172.21.15.47:0/2025522230 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f39480b1a30 con 0x55674335a920 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.313+0000 7f392ffff700 1 -- 172.21.15.47:0/2025522230 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f39480ad290 con 0x55674335a920 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.313+0000 7f39619a9b80 10 client.25378 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.313+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55674338db30 con 0x55674335a920 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.313+0000 7f392ffff700 1 -- 172.21.15.47:0/2025522230 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39480b1cb0 con 0x55674335a920 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.314+0000 7f392ffff700 1 -- 172.21.15.47:0/2025522230 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f3948012070 con 0x55674335a920 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.314+0000 7f39619a9b80 10 client.25378 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.314+0000 7f39619a9b80 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.314+0000 7f39619a9b80 20 client.25378 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.314+0000 7f39619a9b80 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556743390630 0x556743392a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.314+0000 7f39619a9b80 4 client.25378 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:32:50.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.314+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f394400ed70 con 0x556743390630 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.317+0000 7f3954eaf700 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556743390630 0x556743392a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.335+0000 7f392ffff700 10 client.25378 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.335+0000 7f3954eaf700 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556743390630 0x556743392a50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a4847c1559d8184 server_cookie=b2db3a76624c4e68 in_seq=0 out_seq=0 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.341+0000 7f392ffff700 1 -- 172.21.15.47:0/2025522230 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f394400ed70 con 0x556743390630 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.341+0000 7f392ffff700 10 client.25378 handle_command_reply: tid=0 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.341+0000 7f39619a9b80 1 client.25378 shutdown 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 msgr2=0x55674335df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x55674335df30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f39480df300 tx=0x7f3948007ee0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 shutdown_connections 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x556743390630 0x556743392a50 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x55674335df30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743360600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 --2- 172.21.15.47:0/2025522230 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674335d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 >> 172.21.15.47:0/2025522230 conn(0x556743280b70 msgr2=0x55674334e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f392ffff700 0 client.25378 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.343+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 shutdown_connections 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.344+0000 7f39619a9b80 1 -- 172.21.15.47:0/2025522230 wait complete. 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.344+0000 7f39619a9b80 20 client.25378 trim_cache size 0 max 16384 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.420+0000 7f39619a9b80 1 Processor -- start 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.425+0000 7f39619a9b80 1 -- start start 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.425+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.425+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674336a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.425+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 0x55674335c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f3954eaf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f3954eaf700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:45592/0 (socket says 172.21.15.47:45592) 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f394ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674336a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f394ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674336a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:33426/0 (socket says 172.21.15.47:33426) 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f3954eaf700 1 -- 172.21.15.47:0/3605024318 learned_addr learned my addr 172.21.15.47:0/3605024318 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:50.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f393c00b7e0 con 0x55674335a920 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f393c00e060 con 0x55674335a000 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.426+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f393c00be50 con 0x556743354b30 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.428+0000 7f39556b0700 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 0x55674335c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.428+0000 7f3954eaf700 1 -- 172.21.15.47:0/3605024318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 msgr2=0x55674336a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.428+0000 7f3954eaf700 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674336a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.428+0000 7f3954eaf700 1 -- 172.21.15.47:0/3605024318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 msgr2=0x55674335c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.428+0000 7f3954eaf700 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 0x55674335c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.428+0000 7f3954eaf700 1 -- 172.21.15.47:0/3605024318 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556743350f90 con 0x556743354b30 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.430+0000 7f3954eaf700 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3944005490 tx=0x7f3944005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e59fa79f13360e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.431+0000 7f39367fc700 1 -- 172.21.15.47:0/3605024318 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944014040 con 0x556743354b30 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.431+0000 7f39367fc700 1 -- 172.21.15.47:0/3605024318 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3944028070 con 0x556743354b30 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.432+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 msgr2=0x556743354510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.432+0000 7f39619a9b80 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3944005490 tx=0x7f3944005420 comp rx=0 tx=0).stop 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.433+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 shutdown_connections 2024-05-11T12:32:50.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.433+0000 7f39619a9b80 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55674335a920 0x55674335c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.433+0000 7f39619a9b80 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x556743354b30 0x556743354510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.433+0000 7f39619a9b80 1 --2- 172.21.15.47:0/3605024318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x55674336a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.433+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 >> 172.21.15.47:0/3605024318 conn(0x556743280b70 msgr2=0x556743357a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.434+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 shutdown_connections 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.434+0000 7f39619a9b80 1 -- 172.21.15.47:0/3605024318 wait complete. 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.435+0000 7f39619a9b80 1 -- start start 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.435+0000 7f39619a9b80 1 Processor -- start 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.436+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743397b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x556743398080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f39619a9b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x5567433985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f39619a9b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f393c00c520 con 0x556743354b30 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f39619a9b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f393c00d990 con 0x55674335a000 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f39619a9b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f393c00cbf0 con 0x55674335a920 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f394ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x556743398080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f394ffff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x556743398080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:33440/0 (socket says 172.21.15.47:33440) 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.437+0000 7f394ffff700 1 -- 172.21.15.47:0/228165245 learned_addr learned my addr 172.21.15.47:0/228165245 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.438+0000 7f39556b0700 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x5567433985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.438+0000 7f39619a9b80 10 client.0 _ll_register_callbacks cb 0x55674323e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.439+0000 7f394ffff700 1 -- 172.21.15.47:0/228165245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 msgr2=0x5567433985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.439+0000 7f394ffff700 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x5567433985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.439+0000 7f394ffff700 1 -- 172.21.15.47:0/228165245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 msgr2=0x556743397b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.439+0000 7f394ffff700 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743397b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.439+0000 7f394ffff700 1 -- 172.21.15.47:0/228165245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567433f9f20 con 0x55674335a000 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.441+0000 7f394ffff700 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x556743398080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3940009990 tx=0x7f3940000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f9b706d0d8e718 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.442+0000 7f392ffff700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.443+0000 7f392ffff700 1 -- 172.21.15.47:0/228165245 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940030060 con 0x55674335a000 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.443+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3944028070 con 0x55674335a000 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.443+0000 7f392ffff700 1 -- 172.21.15.47:0/228165245 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f3940012070 con 0x55674335a000 2024-05-11T12:32:50.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.443+0000 7f392ffff700 1 -- 172.21.15.47:0/228165245 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f394001a030 con 0x55674335a000 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.447+0000 7f39619a9b80 10 client.25349 fetch_fsmap learned FSMap version 20 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.447+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5567433fb220 con 0x55674335a000 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.453+0000 7f392ffff700 1 -- 172.21.15.47:0/228165245 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3944028070 con 0x55674335a000 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.454+0000 7f392ffff700 1 -- 172.21.15.47:0/228165245 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f394002a070 con 0x55674335a000 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.454+0000 7f39619a9b80 10 client.25349 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.454+0000 7f39619a9b80 10 client.25349 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.454+0000 7f39619a9b80 20 client.25349 populate_metadata read hostname 'smithi047' 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.454+0000 7f39619a9b80 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5567433fb990 0x5567433fddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.454+0000 7f39619a9b80 4 client.25349 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.454+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567432b4320 con 0x5567433fb990 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.455+0000 7f3954eaf700 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5567433fb990 0x5567433fddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.473+0000 7f3954eaf700 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5567433fb990 0x5567433fddb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624e485879a7e02a server_cookie=ebd5729f655e84a6 in_seq=0 out_seq=0 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.473+0000 7f392ffff700 10 client.25349 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:50.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.474+0000 7f392ffff700 1 -- 172.21.15.47:0/228165245 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5567432b4320 con 0x5567433fb990 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.474+0000 7f392ffff700 10 client.25349 handle_command_reply: tid=0 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 client.25349 shutdown 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 msgr2=0x556743398080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x556743398080 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3940009990 tx=0x7f3940000bc0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 shutdown_connections 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5567433fb990 0x5567433fddb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55674335a920 0x5567433985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x556743354b30 0x556743397b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 --2- 172.21.15.47:0/228165245 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55674335a000 0x556743398080 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3940009990 tx=0x7f3940000bc0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.483+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 >> 172.21.15.47:0/228165245 conn(0x556743280b70 msgr2=0x55674335c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.485+0000 7f392ffff700 0 client.25349 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.486+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 shutdown_connections 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.487+0000 7f39619a9b80 1 -- 172.21.15.47:0/228165245 wait complete. 2024-05-11T12:32:50.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.487+0000 7f39619a9b80 20 client.25349 trim_cache size 0 max 16384 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.499+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f39380b29c0 msgr2=0x7f39380b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.499+0000 7f3955eb1700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f39380b29c0 0x7f39380b4e80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f394000d7d0 tx=0x7f3940005fd0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.499+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501076c0 msgr2=0x7f3950112200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.499+0000 7f3955eb1700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501076c0 0x7f3950112200 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f394800ada0 tx=0x7f394800c330 comp rx=0 tx=0).stop 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.500+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 shutdown_connections 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.500+0000 7f3955eb1700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39501076c0 0x7f3950112200 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.500+0000 7f3955eb1700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f39501025e0 0x7f3950117c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.500+0000 7f3955eb1700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f39380b29c0 0x7f39380b4e80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.500+0000 7f3955eb1700 1 --2- 172.21.15.47:0/1076474888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3950102f90 0x7f3950111cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.500+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 >> 172.21.15.47:0/1076474888 conn(0x7f39500fde70 msgr2=0x7f39501001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.508+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 shutdown_connections 2024-05-11T12:32:50.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:50.508+0000 7f3955eb1700 1 -- 172.21.15.47:0/1076474888 wait complete. 2024-05-11T12:32:52.029 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:51 smithi047 ceph-mon[30375]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 758 MiB/s wr, 1.59k op/s 2024-05-11T12:32:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:51 smithi064 ceph-mon[34537]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 758 MiB/s wr, 1.59k op/s 2024-05-11T12:32:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:51 smithi050 ceph-mon[34072]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 758 MiB/s wr, 1.59k op/s 2024-05-11T12:32:52.731 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:32:52.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:32:53.709 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:32:53.983 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:53 smithi047 ceph-mon[30375]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 865 MiB/s wr, 2.09k op/s 2024-05-11T12:32:53.984 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:53 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:32:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:53 smithi064 ceph-mon[34537]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 865 MiB/s wr, 2.09k op/s 2024-05-11T12:32:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:53 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:32:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:53 smithi050 ceph-mon[34072]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 865 MiB/s wr, 2.09k op/s 2024-05-11T12:32:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:53 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:32:56.000 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:55 smithi050 ceph-mon[34072]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 771 MiB/s wr, 2.18k op/s 2024-05-11T12:32:56.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:55 smithi047 ceph-mon[30375]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 771 MiB/s wr, 2.18k op/s 2024-05-11T12:32:56.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:55 smithi064 ceph-mon[34537]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 771 MiB/s wr, 2.18k op/s 2024-05-11T12:32:57.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:32:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:32:57.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:56 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:32:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:32:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:32:57.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:56 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:32:57.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:32:57.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:32:57.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:56 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:32:57.884 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:57 smithi047 ceph-mon[30375]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 681 MiB/s wr, 2.08k op/s 2024-05-11T12:32:58.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:57 smithi050 ceph-mon[34072]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 681 MiB/s wr, 2.08k op/s 2024-05-11T12:32:58.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:57 smithi064 ceph-mon[34537]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 681 MiB/s wr, 2.08k op/s 2024-05-11T12:32:58.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.634+0000 7f07540a4700 1 -- 172.21.15.47:0/811362207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 msgr2=0x7f074c10e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.634+0000 7f07540a4700 1 --2- 172.21.15.47:0/811362207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c10e7e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f074400b0a0 tx=0x7f074400b3b0 comp rx=0 tx=0).stop 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.636+0000 7f07540a4700 1 -- 172.21.15.47:0/811362207 shutdown_connections 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.636+0000 7f07540a4700 1 --2- 172.21.15.47:0/811362207 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f074c10ef10 0x7f074c07b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.636+0000 7f07540a4700 1 --2- 172.21.15.47:0/811362207 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f074c10a7b0 0x7f074c10ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.636+0000 7f07540a4700 1 --2- 172.21.15.47:0/811362207 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c10e7e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.636+0000 7f07540a4700 1 -- 172.21.15.47:0/811362207 >> 172.21.15.47:0/811362207 conn(0x7f074c06e6c0 msgr2=0x7f074c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.644+0000 7f07540a4700 1 -- 172.21.15.47:0/811362207 shutdown_connections 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.644+0000 7f07540a4700 1 -- 172.21.15.47:0/811362207 wait complete. 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 Processor -- start 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 -- start start 2024-05-11T12:32:58.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f074c10a7b0 0x7f074c07b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f074c10ef10 0x7f074c075d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f074c082fc0 con 0x7f074c10ef10 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f074c082e40 con 0x7f074c10a7b0 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.645+0000 7f07540a4700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f074c083140 con 0x7f074c10b160 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.648+0000 7f075163f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.648+0000 7f075163f700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53054/0 (socket says 172.21.15.47:53054) 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.648+0000 7f075163f700 1 -- 172.21.15.47:0/78727490 learned_addr learned my addr 172.21.15.47:0/78727490 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.648+0000 7f0751e40700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f074c10a7b0 0x7f074c07b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.650+0000 7f075163f700 1 -- 172.21.15.47:0/78727490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f074c10a7b0 msgr2=0x7f074c07b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.650+0000 7f075163f700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f074c10a7b0 0x7f074c07b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.650+0000 7f075163f700 1 -- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f074c10ef10 msgr2=0x7f074c075d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:58.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.650+0000 7f075163f700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f074c10ef10 0x7f074c075d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.650+0000 7f075163f700 1 -- 172.21.15.47:0/78727490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c1c5f70 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.651+0000 7f075163f700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c0757a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f074400b070 tx=0x7f0744004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.655+0000 7f0742ffd700 1 -- 172.21.15.47:0/78727490 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744021070 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.655+0000 7f07540a4700 1 -- 172.21.15.47:0/78727490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074c1c6140 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.655+0000 7f07540a4700 1 -- 172.21.15.47:0/78727490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f074c1c6590 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.657+0000 7f0742ffd700 1 -- 172.21.15.47:0/78727490 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0744007e30 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.657+0000 7f0742ffd700 1 -- 172.21.15.47:0/78727490 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07440046f0 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.659+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f074c06fc50 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.664+0000 7f0742ffd700 1 -- 172.21.15.47:0/78727490 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0744007980 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.665+0000 7f0742ffd700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07380b2a90 0x7f07380b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.666+0000 7f0742ffd700 1 -- 172.21.15.47:0/78727490 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f07440e30a0 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.672+0000 7f0751e40700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07380b2a90 0x7f07380b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.672+0000 7f0742ffd700 1 -- 172.21.15.47:0/78727490 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f07440aada0 con 0x7f074c10b160 2024-05-11T12:32:58.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:58.673+0000 7f0751e40700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07380b2a90 0x7f07380b4f50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0748009c60 tx=0x7f0748009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:32:59.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.374+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f074c076cd0 con 0x7f074c10b160 2024-05-11T12:32:59.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.375+0000 7f0742ffd700 1 -- 172.21.15.47:0/78727490 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f07440aa6a0 con 0x7f074c10b160 2024-05-11T12:32:59.379 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:32:59.379 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:32:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.389+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07380b2a90 msgr2=0x7f07380b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.389+0000 7f0740ff9700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07380b2a90 0x7f07380b4f50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0748009c60 tx=0x7f0748009260 comp rx=0 tx=0).stop 2024-05-11T12:32:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.389+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 msgr2=0x7f074c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:32:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.390+0000 7f0740ff9700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c0757a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f074400b070 tx=0x7f0744004c60 comp rx=0 tx=0).stop 2024-05-11T12:32:59.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.392+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 shutdown_connections 2024-05-11T12:32:59.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.392+0000 7f0740ff9700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f074c10ef10 0x7f074c075d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.392+0000 7f0740ff9700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f07380b2a90 0x7f07380b4f50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.392+0000 7f0740ff9700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f074c10a7b0 0x7f074c07b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.392+0000 7f0740ff9700 1 --2- 172.21.15.47:0/78727490 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f074c10b160 0x7f074c0757a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:32:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.392+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 >> 172.21.15.47:0/78727490 conn(0x7f074c06e6c0 msgr2=0x7f074c0744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:32:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.398+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 shutdown_connections 2024-05-11T12:32:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:32:59.402+0000 7f0740ff9700 1 -- 172.21.15.47:0/78727490 wait complete. 2024-05-11T12:32:59.410 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:33:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:59 smithi050 ceph-mon[34072]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 834 MiB/s wr, 2.32k op/s 2024-05-11T12:33:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:32:59 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/78727490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:33:00.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:59 smithi047 ceph-mon[30375]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 834 MiB/s wr, 2.32k op/s 2024-05-11T12:33:00.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:32:59 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/78727490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:33:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:59 smithi064 ceph-mon[34537]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 834 MiB/s wr, 2.32k op/s 2024-05-11T12:33:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:32:59 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/78727490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:33:01.285 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:00 smithi047 ceph-mon[30375]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 691 MiB/s wr, 1.95k op/s 2024-05-11T12:33:01.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:00 smithi064 ceph-mon[34537]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 691 MiB/s wr, 1.95k op/s 2024-05-11T12:33:01.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:00 smithi050 ceph-mon[34072]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 691 MiB/s wr, 1.95k op/s 2024-05-11T12:33:01.828 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:33:02.855 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:33:03.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:03 smithi064 ceph-mon[34537]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 789 MiB/s wr, 2.16k op/s 2024-05-11T12:33:03.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:03 smithi050 ceph-mon[34072]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 789 MiB/s wr, 2.16k op/s 2024-05-11T12:33:03.791 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:03 smithi047 ceph-mon[30375]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 789 MiB/s wr, 2.16k op/s 2024-05-11T12:33:05.359 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:05 smithi047 ceph-mon[30375]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 700 MiB/s wr, 1.79k op/s 2024-05-11T12:33:05.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:05 smithi064 ceph-mon[34537]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 700 MiB/s wr, 1.79k op/s 2024-05-11T12:33:05.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:05 smithi050 ceph-mon[34072]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 700 MiB/s wr, 1.79k op/s 2024-05-11T12:33:07.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:07 smithi047 ceph-mon[30375]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 691 MiB/s wr, 1.50k op/s 2024-05-11T12:33:07.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:07 smithi064 ceph-mon[34537]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 691 MiB/s wr, 1.50k op/s 2024-05-11T12:33:07.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:07 smithi050 ceph-mon[34072]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 691 MiB/s wr, 1.50k op/s 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.942+0000 7f2d8a338700 1 -- 172.21.15.47:0/1575285607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 msgr2=0x7f2d841168f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.942+0000 7f2d8a338700 1 --2- 172.21.15.47:0/1575285607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841168f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009990 tx=0x7f2d7c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.946+0000 7f2d8a338700 1 -- 172.21.15.47:0/1575285607 shutdown_connections 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.946+0000 7f2d8a338700 1 --2- 172.21.15.47:0/1575285607 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841168f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.946+0000 7f2d8a338700 1 --2- 172.21.15.47:0/1575285607 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d8410e960 0x7f2d8410ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.946+0000 7f2d8a338700 1 --2- 172.21.15.47:0/1575285607 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d84109a90 0x7f2d84109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.946+0000 7f2d8a338700 1 -- 172.21.15.47:0/1575285607 >> 172.21.15.47:0/1575285607 conn(0x7f2d840792e0 msgr2=0x7f2d84108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.951+0000 7f2d8a338700 1 -- 172.21.15.47:0/1575285607 shutdown_connections 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.951+0000 7f2d8a338700 1 -- 172.21.15.47:0/1575285607 wait complete. 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.953+0000 7f2d8a338700 1 Processor -- start 2024-05-11T12:33:08.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.953+0000 7f2d8a338700 1 -- start start 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.954+0000 7f2d8a338700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d84109a90 0x7f2d841a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.954+0000 7f2d8a338700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.954+0000 7f2d8a338700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d8410e960 0x7f2d841a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.955+0000 7f2d8a338700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84119c90 con 0x7f2d8410a450 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.955+0000 7f2d8a338700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84119b10 con 0x7f2d84109a90 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.955+0000 7f2d8a338700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d84119e10 con 0x7f2d8410e960 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.957+0000 7f2d89b37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d8410e960 0x7f2d841a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.957+0000 7f2d89336700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d84109a90 0x7f2d841a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.958+0000 7f2d88b35700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.958+0000 7f2d88b35700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841a0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41536/0 (socket says 172.21.15.47:41536) 2024-05-11T12:33:08.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.958+0000 7f2d88b35700 1 -- 172.21.15.47:0/2959748702 learned_addr learned my addr 172.21.15.47:0/2959748702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.959+0000 7f2d88b35700 1 -- 172.21.15.47:0/2959748702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d8410e960 msgr2=0x7f2d841a4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.959+0000 7f2d88b35700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d8410e960 0x7f2d841a4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.959+0000 7f2d88b35700 1 -- 172.21.15.47:0/2959748702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d84109a90 msgr2=0x7f2d841a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.959+0000 7f2d88b35700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d84109a90 0x7f2d841a00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.959+0000 7f2d88b35700 1 -- 172.21.15.47:0/2959748702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d841a50d0 con 0x7f2d8410a450 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.959+0000 7f2d88b35700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841a0600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d950 tx=0x7f2d7400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.960+0000 7f2d7a7fc700 1 -- 172.21.15.47:0/2959748702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74016070 con 0x7f2d8410a450 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.960+0000 7f2d7a7fc700 1 -- 172.21.15.47:0/2959748702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d74010460 con 0x7f2d8410a450 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.960+0000 7f2d7a7fc700 1 -- 172.21.15.47:0/2959748702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400f600 con 0x7f2d8410a450 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.960+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d841a53c0 con 0x7f2d8410a450 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.960+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8407bda0 con 0x7f2d8410a450 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.981+0000 7f2d95e30b80 1 Processor -- start 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.994+0000 7f2d7a7fc700 1 -- 172.21.15.47:0/2959748702 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d74019070 con 0x7f2d8410a450 2024-05-11T12:33:08.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.994+0000 7f2d95e30b80 1 -- start start 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.995+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffa9a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.995+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffa9e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.995+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 0x55a2ffa9f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.995+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a2ff9ec860 con 0x55a2ffa9aba0 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.995+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a2ff98b7c0 con 0x55a2ffa9a280 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.995+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a2ff9e8820 con 0x55a2ffa9ed50 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.996+0000 7f2d89336700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffa9e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.996+0000 7f2d89336700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffa9e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41538/0 (socket says 172.21.15.47:41538) 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.996+0000 7f2d89336700 1 -- 172.21.15.47:0/839187512 learned_addr learned my addr 172.21.15.47:0/839187512 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.996+0000 7f2d89b37700 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffa9a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d89336700 1 -- 172.21.15.47:0/839187512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 msgr2=0x55a2ffa9f190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d89336700 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 0x55a2ffa9f190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d89336700 1 -- 172.21.15.47:0/839187512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 msgr2=0x55a2ffa9a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d7a7fc700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2d700b2a10 0x7f2d700b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d89336700 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffa9a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d89336700 1 -- 172.21.15.47:0/839187512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2ff9398d0 con 0x55a2ffa9aba0 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d89b37700 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffa9a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.997+0000 7f2d7a7fc700 1 -- 172.21.15.47:0/2959748702 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2d740e1d70 con 0x7f2d8410a450 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.998+0000 7f2d89336700 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffa9e810 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009710 tx=0x7f2d80007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfb8be0634bb765 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.998+0000 7f2d6f7fe700 1 -- 172.21.15.47:0/839187512 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80004bb0 con 0x55a2ffa9aba0 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.998+0000 7f2d6f7fe700 1 -- 172.21.15.47:0/839187512 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d80004d50 con 0x55a2ffa9aba0 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:07.998+0000 7f2d6f7fe700 1 -- 172.21.15.47:0/839187512 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80014660 con 0x55a2ffa9aba0 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.001+0000 7f2d95e30b80 1 -- 172.21.15.47:0/839187512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 msgr2=0x55a2ffa9e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.001+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffa9e810 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009710 tx=0x7f2d80007500 comp rx=0 tx=0).stop 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.002+0000 7f2d95e30b80 1 -- 172.21.15.47:0/839187512 shutdown_connections 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.002+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 0x55a2ffa9f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.002+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffa9a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.002+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/839187512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffa9e810 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.002+0000 7f2d95e30b80 1 -- 172.21.15.47:0/839187512 >> 172.21.15.47:0/839187512 conn(0x55a2ff9c6e60 msgr2=0x55a2ffa94cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.002+0000 7f2d88b35700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2d700b2a10 0x7f2d700b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.006+0000 7f2d95e30b80 1 -- 172.21.15.47:0/839187512 shutdown_connections 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.007+0000 7f2d88b35700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2d700b2a10 0x7f2d700b4ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d740a8fc0 tx=0x7f2d7400df80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.007+0000 7f2d95e30b80 1 -- 172.21.15.47:0/839187512 wait complete. 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.008+0000 7f2d95e30b80 1 -- start start 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.008+0000 7f2d95e30b80 1 Processor -- start 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.009+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.010+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaa35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.010+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.010+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55a2ffaa6fb0 con 0x55a2ffa9aba0 2024-05-11T12:33:08.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.010+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55a2ff952c30 con 0x55a2ffa9a280 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.010+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55a2ffaa7130 con 0x55a2ffa9ed50 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.010+0000 7f2d95e30b80 10 client.0 _ll_register_callbacks cb 0x55a2ff984920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.020+0000 7f2d89b37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.020+0000 7f2d89b37700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.020+0000 7f2d89b37700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:48304/0 (socket says 172.21.15.47:48304) 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.020+0000 7f2d89b37700 1 -- 172.21.15.47:0/3883367271 learned_addr learned my addr 172.21.15.47:0/3883367271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.023+0000 7f2d89b37700 1 -- 172.21.15.47:0/3883367271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 msgr2=0x55a2ffaa3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.023+0000 7f2d89b37700 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.023+0000 7f2d89b37700 1 -- 172.21.15.47:0/3883367271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 msgr2=0x55a2ffaa35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.023+0000 7f2d89b37700 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaa35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.023+0000 7f2d89b37700 1 -- 172.21.15.47:0/3883367271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ff988690 con 0x55a2ffa9a280 2024-05-11T12:33:08.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.032+0000 7f2d89b37700 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa6250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0037a0 tx=0x7f2d7c00b540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f633cf6bae31ac8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.032+0000 7f2d6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.033+0000 7f2d6cff9700 1 -- 172.21.15.47:0/3883367271 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c0052c0 con 0x55a2ffa9a280 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.033+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d80004d50 con 0x55a2ffa9a280 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.033+0000 7f2d6cff9700 1 -- 172.21.15.47:0/3883367271 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2d7c005460 con 0x55a2ffa9a280 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.033+0000 7f2d6cff9700 1 -- 172.21.15.47:0/3883367271 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2d7c005660 con 0x55a2ffa9a280 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.037+0000 7f2d95e30b80 10 client.25364 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.037+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a2ffab1990 con 0x55a2ffa9a280 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.038+0000 7f2d6cff9700 1 -- 172.21.15.47:0/3883367271 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d80004d50 con 0x55a2ffa9a280 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.039+0000 7f2d6cff9700 1 -- 172.21.15.47:0/3883367271 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2d7c024070 con 0x55a2ffa9a280 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.044+0000 7f2d95e30b80 10 client.25364 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.044+0000 7f2d95e30b80 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:08.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.044+0000 7f2d95e30b80 20 client.25364 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.044+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffab4390 0x55a2ffab6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.044+0000 7f2d95e30b80 4 client.25364 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.044+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2ff9fa240 con 0x55a2ffab4390 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.047+0000 7f2d89336700 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffab4390 0x55a2ffab6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.062+0000 7f2d89336700 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffab4390 0x55a2ffab6810 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a222c66f299adc96 server_cookie=971a8975c20584a6 in_seq=0 out_seq=0 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.062+0000 7f2d6cff9700 10 client.25364 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.068+0000 7f2d6cff9700 1 -- 172.21.15.47:0/3883367271 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a2ff9fa240 con 0x55a2ffab4390 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.068+0000 7f2d6cff9700 10 client.25364 handle_command_reply: tid=0 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.068+0000 7f2d95e30b80 1 client.25364 shutdown 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.068+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 msgr2=0x55a2ffaa6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.069+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa6250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0037a0 tx=0x7f2d7c00b540 comp rx=0 tx=0).stop 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.074+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 shutdown_connections 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.074+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffab4390 0x55a2ffab6810 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.074+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.074+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa6250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.074+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3883367271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaa35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.074+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 >> 172.21.15.47:0/3883367271 conn(0x55a2ff9c6e60 msgr2=0x55a2ffa94bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.074+0000 7f2d6cff9700 0 client.25364 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.077+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 shutdown_connections 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.077+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3883367271 wait complete. 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.084+0000 7f2d95e30b80 20 client.25364 trim_cache size 0 max 16384 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.130+0000 7f2d95e30b80 1 Processor -- start 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.133+0000 7f2d95e30b80 1 -- start start 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.133+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.134+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9aba0 0x55a2ffab00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.134+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.134+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6800b6d0 con 0x55a2ffa9a280 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.134+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6800df50 con 0x55a2ffa9ed50 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.134+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6800bd40 con 0x55a2ffa9aba0 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.135+0000 7f2d89336700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9aba0 0x55a2ffab00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.135+0000 7f2d89336700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9aba0 0x55a2ffab00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40740/0 (socket says 172.21.15.47:40740) 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.135+0000 7f2d89336700 1 -- 172.21.15.47:0/3734014393 learned_addr learned my addr 172.21.15.47:0/3734014393 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.135+0000 7f2d89b37700 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.136+0000 7f2d89336700 1 -- 172.21.15.47:0/3734014393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9ed50 msgr2=0x55a2ffaa3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.136+0000 7f2d89336700 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.136+0000 7f2d89336700 1 -- 172.21.15.47:0/3734014393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9a280 msgr2=0x55a2ffaa0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.136+0000 7f2d89336700 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.136+0000 7f2d89336700 1 -- 172.21.15.47:0/3734014393 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2ffab6a10 con 0x55a2ffa9aba0 2024-05-11T12:33:08.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.138+0000 7f2d89336700 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9aba0 0x55a2ffab00d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009710 tx=0x7f2d800059a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=665b42004928c53c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.139+0000 7f2d6effd700 1 -- 172.21.15.47:0/3734014393 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8002b070 con 0x55a2ffa9aba0 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.142+0000 7f2d6effd700 1 -- 172.21.15.47:0/3734014393 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d80024070 con 0x55a2ffa9aba0 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.142+0000 7f2d6effd700 1 -- 172.21.15.47:0/3734014393 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800209f0 con 0x55a2ffa9aba0 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.144+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3734014393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9aba0 msgr2=0x55a2ffab00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.144+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9aba0 0x55a2ffab00d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009710 tx=0x7f2d800059a0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.149+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3734014393 shutdown_connections 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.149+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9ed50 0x55a2ffaa3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.149+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9a280 0x55a2ffaa0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.149+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/3734014393 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9aba0 0x55a2ffab00d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.149+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3734014393 >> 172.21.15.47:0/3734014393 conn(0x55a2ff9c6e60 msgr2=0x55a2ffaa07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.151+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3734014393 shutdown_connections 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.153+0000 7f2d95e30b80 1 -- 172.21.15.47:0/3734014393 wait complete. 2024-05-11T12:33:08.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.153+0000 7f2d95e30b80 1 -- start start 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.154+0000 7f2d95e30b80 1 Processor -- start 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.155+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9a280 0x55a2ffaac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.155+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaac990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.155+0000 7f2d95e30b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9ed50 0x55a2ffabb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.155+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6800c410 con 0x55a2ffa9ed50 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.155+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6800d880 con 0x55a2ffa9aba0 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.155+0000 7f2d95e30b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6800cae0 con 0x55a2ffa9a280 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.155+0000 7f2d95e30b80 10 client.0 _ll_register_callbacks cb 0x55a2ff984920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.163+0000 7f2d89336700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.163+0000 7f2d89b37700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9a280 0x55a2ffaac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.164+0000 7f2d89336700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaac990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:48334/0 (socket says 172.21.15.47:48334) 2024-05-11T12:33:08.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.164+0000 7f2d89336700 1 -- 172.21.15.47:0/460461667 learned_addr learned my addr 172.21.15.47:0/460461667 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.169+0000 7f2d89b37700 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9ed50 0x55a2ffabb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.175+0000 7f2d89336700 1 -- 172.21.15.47:0/460461667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9a280 msgr2=0x55a2ffaac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.175+0000 7f2d89336700 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9a280 0x55a2ffaac450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.175+0000 7f2d89336700 1 -- 172.21.15.47:0/460461667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9ed50 msgr2=0x55a2ffabb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.175+0000 7f2d89336700 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9ed50 0x55a2ffabb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.175+0000 7f2d89336700 1 -- 172.21.15.47:0/460461667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2ffabb6d0 con 0x55a2ffa9aba0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.193+0000 7f2d89336700 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaac990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55a2ffa95b50 tx=0x7f2d800254b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c968c936d75d4c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.194+0000 7f2d6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.203+0000 7f2d6cff9700 1 -- 172.21.15.47:0/460461667 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8002b040 con 0x55a2ffa9aba0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.203+0000 7f2d6cff9700 1 -- 172.21.15.47:0/460461667 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f2d80025820 con 0x55a2ffa9aba0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.203+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d80024070 con 0x55a2ffa9aba0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.204+0000 7f2d6cff9700 1 -- 172.21.15.47:0/460461667 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2d80006000 con 0x55a2ffa9aba0 2024-05-11T12:33:08.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.204+0000 7f2d95e30b80 10 client.25370 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.204+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a2ffb1e550 con 0x55a2ffa9aba0 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d6cff9700 1 -- 172.21.15.47:0/460461667 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d80025a20 con 0x55a2ffa9aba0 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d6cff9700 1 -- 172.21.15.47:0/460461667 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f2d80020410 con 0x55a2ffa9aba0 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d95e30b80 10 client.25370 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d95e30b80 10 client.25370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d95e30b80 20 client.25370 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffb1ea30 0x55a2ffb20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d95e30b80 4 client.25370 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.205+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2d80024070 con 0x55a2ffb1ea30 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.210+0000 7f2d89b37700 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffb1ea30 0x55a2ffb20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.213+0000 7f2d6cff9700 10 client.25370 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.213+0000 7f2d89b37700 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffb1ea30 0x55a2ffb20e50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6cc09453fc68ae server_cookie=b1e3651048756bee in_seq=0 out_seq=0 2024-05-11T12:33:08.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.216+0000 7f2d6cff9700 1 -- 172.21.15.47:0/460461667 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2d80024070 con 0x55a2ffb1ea30 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.216+0000 7f2d6cff9700 10 client.25370 handle_command_reply: tid=0 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.216+0000 7f2d95e30b80 1 client.25370 shutdown 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9aba0 msgr2=0x55a2ffaac990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaac990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55a2ffa95b50 tx=0x7f2d800254b0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 shutdown_connections 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55a2ffb1ea30 0x55a2ffb20e50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55a2ffa9ed50 0x55a2ffabb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55a2ffa9a280 0x55a2ffaac450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 --2- 172.21.15.47:0/460461667 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55a2ffa9aba0 0x55a2ffaac990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 >> 172.21.15.47:0/460461667 conn(0x55a2ff9c6e60 msgr2=0x55a2ffa99920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d6cff9700 0 client.25370 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 shutdown_connections 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.219+0000 7f2d95e30b80 1 -- 172.21.15.47:0/460461667 wait complete. 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.220+0000 7f2d95e30b80 20 client.25370 trim_cache size 0 max 16384 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2d700b2a10 msgr2=0x7f2d700b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2d700b2a10 0x7f2d700b4ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d740a8fc0 tx=0x7f2d7400df80 comp rx=0 tx=0).stop 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 msgr2=0x7f2d841a0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841a0600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400d950 tx=0x7f2d7400dd10 comp rx=0 tx=0).stop 2024-05-11T12:33:08.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 shutdown_connections 2024-05-11T12:33:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f2d8410e960 0x7f2d841a4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f2d700b2a10 0x7f2d700b4ed0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2d84109a90 0x7f2d841a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 --2- 172.21.15.47:0/2959748702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d8410a450 0x7f2d841a0600 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.222+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 >> 172.21.15.47:0/2959748702 conn(0x7f2d840792e0 msgr2=0x7f2d840798a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.223+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 shutdown_connections 2024-05-11T12:33:08.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:08.223+0000 7f2d8a338700 1 -- 172.21.15.47:0/2959748702 wait complete. 2024-05-11T12:33:09.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:09 smithi064 ceph-mon[34537]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 844 MiB/s wr, 1.68k op/s 2024-05-11T12:33:09.640 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:09 smithi047 ceph-mon[30375]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 844 MiB/s wr, 1.68k op/s 2024-05-11T12:33:09.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:09 smithi050 ceph-mon[34072]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 844 MiB/s wr, 1.68k op/s 2024-05-11T12:33:11.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:11 smithi064 ceph-mon[34537]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 798 MiB/s wr, 1.79k op/s 2024-05-11T12:33:11.686 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:33:11.687 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:33:11.698 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:11 smithi050 ceph-mon[34072]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 798 MiB/s wr, 1.79k op/s 2024-05-11T12:33:11.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:11 smithi047 ceph-mon[30375]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 798 MiB/s wr, 1.79k op/s 2024-05-11T12:33:12.670 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:33:13.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:13 smithi050 ceph-mon[34072]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 821 MiB/s wr, 1.96k op/s 2024-05-11T12:33:13.703 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:13 smithi047 ceph-mon[30375]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 821 MiB/s wr, 1.96k op/s 2024-05-11T12:33:13.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:13 smithi064 ceph-mon[34537]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 821 MiB/s wr, 1.96k op/s 2024-05-11T12:33:15.689 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:15 smithi047 ceph-mon[30375]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 736 MiB/s wr, 1.68k op/s 2024-05-11T12:33:15.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:15 smithi050 ceph-mon[34072]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 736 MiB/s wr, 1.68k op/s 2024-05-11T12:33:15.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:15 smithi064 ceph-mon[34537]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 736 MiB/s wr, 1.68k op/s 2024-05-11T12:33:17.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:17 smithi050 ceph-mon[34072]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 747 MiB/s wr, 1.54k op/s 2024-05-11T12:33:17.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:17 smithi047 ceph-mon[30375]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 747 MiB/s wr, 1.54k op/s 2024-05-11T12:33:17.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:17 smithi064 ceph-mon[34537]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 747 MiB/s wr, 1.54k op/s 2024-05-11T12:33:18.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.696+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3934979681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4c111d30 msgr2=0x7f0a4c10a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:18.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.696+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3934979681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4c111d30 0x7f0a4c10a550 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009990 tx=0x7f0a3c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.700+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3934979681 shutdown_connections 2024-05-11T12:33:18.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.700+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3934979681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c11cd30 0x7f0a4c11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.700+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3934979681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4c111d30 0x7f0a4c10a550 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.700+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3934979681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a4c10aa90 0x7f0a4c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.700+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3934979681 >> 172.21.15.47:0/3934979681 conn(0x7f0a4c06e6c0 msgr2=0x7f0a4c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:18.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.703+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3934979681 shutdown_connections 2024-05-11T12:33:18.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.706+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3934979681 wait complete. 2024-05-11T12:33:18.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.707+0000 7f0a50ce1700 1 Processor -- start 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.709+0000 7f0a50ce1700 1 -- start start 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.709+0000 7f0a50ce1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a4c10aa90 0x7f0a4c11c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.709+0000 7f0a50ce1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c111d30 0x7f0a4c11c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.709+0000 7f0a50ce1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4c11cd30 0x7f0a4c1163a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.709+0000 7f0a50ce1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c112c70 con 0x7f0a4c11cd30 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.709+0000 7f0a50ce1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c112af0 con 0x7f0a4c10aa90 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.709+0000 7f0a50ce1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c112df0 con 0x7f0a4c111d30 2024-05-11T12:33:18.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c111d30 0x7f0a4c11c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c111d30 0x7f0a4c11c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39276/0 (socket says 172.21.15.47:39276) 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 -- 172.21.15.47:0/3539206591 learned_addr learned my addr 172.21.15.47:0/3539206591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 -- 172.21.15.47:0/3539206591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a4c10aa90 msgr2=0x7f0a4c11c1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a4c10aa90 0x7f0a4c11c1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 -- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4c11cd30 msgr2=0x7f0a4c1163a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4c11cd30 0x7f0a4c1163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.710+0000 7f0a49d9b700 1 -- 172.21.15.47:0/3539206591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c116ca0 con 0x7f0a4c111d30 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.711+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c111d30 0x7f0a4c11c710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400ef10 tx=0x7f0a4400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:18.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.711+0000 7f0a3b7fe700 1 -- 172.21.15.47:0/3539206591 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400cad0 con 0x7f0a4c111d30 2024-05-11T12:33:18.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.711+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c116f30 con 0x7f0a4c111d30 2024-05-11T12:33:18.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.712+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c1bde70 con 0x7f0a4c111d30 2024-05-11T12:33:18.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.712+0000 7f0a3b7fe700 1 -- 172.21.15.47:0/3539206591 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a44019070 con 0x7f0a4c111d30 2024-05-11T12:33:18.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.712+0000 7f0a3b7fe700 1 -- 172.21.15.47:0/3539206591 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440146c0 con 0x7f0a4c111d30 2024-05-11T12:33:18.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.720+0000 7f0a3b7fe700 1 -- 172.21.15.47:0/3539206591 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0a44012050 con 0x7f0a4c111d30 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.721+0000 7f0a5b577b80 1 Processor -- start 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.725+0000 7f0a5b577b80 1 -- start start 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.726+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.726+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533ada55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.726+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.726+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56533acf78e0 con 0x56533ada5ae0 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.726+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56533ac96840 con 0x56533ada5120 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.726+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56533acf38a0 con 0x56533adaab50 2024-05-11T12:33:18.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.727+0000 7f0a4ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.727+0000 7f0a49d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.727+0000 7f0a4ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaaf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39278/0 (socket says 172.21.15.47:39278) 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.727+0000 7f0a4ad9d700 1 -- 172.21.15.47:0/2864206150 learned_addr learned my addr 172.21.15.47:0/2864206150 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.727+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533ada55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.727+0000 7f0a49d9b700 1 -- 172.21.15.47:0/2864206150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 msgr2=0x56533adaaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.728+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.728+0000 7f0a49d9b700 1 -- 172.21.15.47:0/2864206150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 msgr2=0x56533ada55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:18.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.728+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533ada55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.728+0000 7f0a49d9b700 1 -- 172.21.15.47:0/2864206150 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533ac44950 con 0x56533ada5ae0 2024-05-11T12:33:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.728+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb1910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009c10 tx=0x7f0a44013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2216c449838cf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:18.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.729+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533ada55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:33:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.729+0000 7f0a38ff9700 1 -- 172.21.15.47:0/2864206150 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440e6e60 con 0x56533ada5ae0 2024-05-11T12:33:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.733+0000 7f0a3b7fe700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a340b2a90 0x7f0a340b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.734+0000 7f0a3b7fe700 1 -- 172.21.15.47:0/3539206591 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0a440e3240 con 0x7f0a4c111d30 2024-05-11T12:33:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.734+0000 7f0a38ff9700 1 -- 172.21.15.47:0/2864206150 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a440e62e0 con 0x56533ada5ae0 2024-05-11T12:33:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.735+0000 7f0a5b577b80 1 -- 172.21.15.47:0/2864206150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 msgr2=0x56533adb1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.735+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb1910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009c10 tx=0x7f0a44013000 comp rx=0 tx=0).stop 2024-05-11T12:33:18.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.742+0000 7f0a5b577b80 1 -- 172.21.15.47:0/2864206150 shutdown_connections 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.742+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb1910 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.742+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.742+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/2864206150 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533ada55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.743+0000 7f0a5b577b80 1 -- 172.21.15.47:0/2864206150 >> 172.21.15.47:0/2864206150 conn(0x56533acd1ee0 msgr2=0x56533ad9fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.743+0000 7f0a4ad9d700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a340b2a90 0x7f0a340b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.743+0000 7f0a5b577b80 1 -- 172.21.15.47:0/2864206150 shutdown_connections 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.743+0000 7f0a5b577b80 1 -- 172.21.15.47:0/2864206150 wait complete. 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.744+0000 7f0a5b577b80 1 -- start start 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.744+0000 7f0a4ad9d700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a340b2a90 0x7f0a340b4f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0a40006fd0 tx=0x7f0a40009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.744+0000 7f0a5b577b80 1 Processor -- start 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.745+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.745+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.745+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.745+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56533adb1f10 con 0x56533ada5ae0 2024-05-11T12:33:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.745+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x56533ac5dcb0 con 0x56533ada5120 2024-05-11T12:33:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.745+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x56533adb2090 con 0x56533adaab50 2024-05-11T12:33:18.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.745+0000 7f0a5b577b80 10 client.0 _ll_register_callbacks cb 0x56533ac8f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.747+0000 7f0a4a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.747+0000 7f0a4a59c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:46010/0 (socket says 172.21.15.47:46010) 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.747+0000 7f0a4ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.747+0000 7f0a4a59c700 1 -- 172.21.15.47:0/628214194 learned_addr learned my addr 172.21.15.47:0/628214194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.752+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.752+0000 7f0a49d9b700 1 -- 172.21.15.47:0/628214194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 msgr2=0x56533adaed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.752+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.752+0000 7f0a49d9b700 1 -- 172.21.15.47:0/628214194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 msgr2=0x56533adb14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.752+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.752+0000 7f0a49d9b700 1 -- 172.21.15.47:0/628214194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533ac93710 con 0x56533ada5ae0 2024-05-11T12:33:18.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.758+0000 7f0a49d9b700 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adae840 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009c10 tx=0x7f0a440e6b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52bc7437d9f39ba7 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:18.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.758+0000 7f0a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:33:18.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.763+0000 7f0a31ffb700 1 -- 172.21.15.47:0/628214194 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440e6e60 con 0x56533ada5ae0 2024-05-11T12:33:18.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.763+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a440e62e0 con 0x56533ada5ae0 2024-05-11T12:33:18.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.763+0000 7f0a31ffb700 1 -- 172.21.15.47:0/628214194 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0a440e60e0 con 0x56533ada5ae0 2024-05-11T12:33:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.766+0000 7f0a31ffb700 1 -- 172.21.15.47:0/628214194 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0a440b1e60 con 0x56533ada5ae0 2024-05-11T12:33:18.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.768+0000 7f0a5b577b80 10 client.15795 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:18.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.768+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56533adb1f10 con 0x56533ada5ae0 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a31ffb700 1 -- 172.21.15.47:0/628214194 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a440e62e0 con 0x56533ada5ae0 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a31ffb700 1 -- 172.21.15.47:0/628214194 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0a440b3100 con 0x56533ada5ae0 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a5b577b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a5b577b80 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a5b577b80 20 client.15795 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533adbeb40 0x56533adc0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a5b577b80 4 client.15795 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:33:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.769+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56533ad052c0 con 0x56533adbeb40 2024-05-11T12:33:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.770+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533adbeb40 0x56533adc0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.774+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533adbeb40 0x56533adc0f30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2c28147b464c050 server_cookie=89bbc37de552553f in_seq=0 out_seq=0 2024-05-11T12:33:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.774+0000 7f0a31ffb700 10 client.15795 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.781+0000 7f0a31ffb700 1 -- 172.21.15.47:0/628214194 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56533ad052c0 con 0x56533adbeb40 2024-05-11T12:33:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.781+0000 7f0a31ffb700 10 client.15795 handle_command_reply: tid=0 2024-05-11T12:33:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.781+0000 7f0a5b577b80 1 client.15795 shutdown 2024-05-11T12:33:18.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.781+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 msgr2=0x56533adae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.781+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adae840 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009c10 tx=0x7f0a440e6b40 comp rx=0 tx=0).stop 2024-05-11T12:33:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 shutdown_connections 2024-05-11T12:33:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533adbeb40 0x56533adc0f30 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adaed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/628214194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adae840 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 >> 172.21.15.47:0/628214194 conn(0x56533acd1ee0 msgr2=0x56533ad9a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a31ffb700 0 client.15795 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 shutdown_connections 2024-05-11T12:33:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.782+0000 7f0a5b577b80 1 -- 172.21.15.47:0/628214194 wait complete. 2024-05-11T12:33:18.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.783+0000 7f0a5b577b80 20 client.15795 trim_cache size 0 max 16384 2024-05-11T12:33:18.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.856+0000 7f0a5b577b80 1 Processor -- start 2024-05-11T12:33:18.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.857+0000 7f0a5b577b80 1 -- start start 2024-05-11T12:33:18.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.858+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5120 0x56533adbb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.858+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533ada5ae0 0x56533ada8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.858+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533adaab50 0x56533ada4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.858+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c00b790 con 0x56533ada5120 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.858+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c00e010 con 0x56533adaab50 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.858+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c00be00 con 0x56533ada5ae0 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.858+0000 7f0a4a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5120 0x56533adbb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.859+0000 7f0a4a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5120 0x56533adbb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38704/0 (socket says 172.21.15.47:38704) 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.859+0000 7f0a4a59c700 1 -- 172.21.15.47:0/1378671458 learned_addr learned my addr 172.21.15.47:0/1378671458 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.859+0000 7f0a4a59c700 1 -- 172.21.15.47:0/1378671458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533ada5ae0 msgr2=0x56533ada8e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.859+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/1378671458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533ada5ae0 0x56533ada8e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.859+0000 7f0a4a59c700 1 -- 172.21.15.47:0/1378671458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533adaab50 msgr2=0x56533ada4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.859+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/1378671458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533adaab50 0x56533ada4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.859+0000 7f0a4a59c700 1 -- 172.21.15.47:0/1378671458 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533ad9c610 con 0x56533ada5120 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.860+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/1378671458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5120 0x56533adbb050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009290 tx=0x7f0a3c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3306227c636c6520 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:18.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.861+0000 7f0a33fff700 1 -- 172.21.15.47:0/1378671458 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c00b7f0 con 0x56533ada5120 2024-05-11T12:33:18.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.861+0000 7f0a33fff700 1 -- 172.21.15.47:0/1378671458 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a3c02d070 con 0x56533ada5120 2024-05-11T12:33:18.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.862+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1378671458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5120 msgr2=0x56533adbb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:18.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.862+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1378671458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5120 0x56533adbb050 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009290 tx=0x7f0a3c009ca0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.867+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1378671458 shutdown_connections 2024-05-11T12:33:18.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.867+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1378671458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533adaab50 0x56533ada4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.867+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1378671458 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5120 0x56533adbb050 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.867+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1378671458 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533ada5ae0 0x56533ada8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.867+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1378671458 >> 172.21.15.47:0/1378671458 conn(0x56533acd1ee0 msgr2=0x56533ada4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.885+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1378671458 shutdown_connections 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.886+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1378671458 wait complete. 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.886+0000 7f0a5b577b80 1 -- start start 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.887+0000 7f0a5b577b80 1 Processor -- start 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.887+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.887+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.887+0000 7f0a5b577b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adb5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.887+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c00c4d0 con 0x56533ada5ae0 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.887+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c00d940 con 0x56533ada5120 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.887+0000 7f0a5b577b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c00cba0 con 0x56533adaab50 2024-05-11T12:33:18.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.888+0000 7f0a5b577b80 10 client.0 _ll_register_callbacks cb 0x56533ac8f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:18.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.888+0000 7f0a4ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adb5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.888+0000 7f0a4ad9d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adb5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39288/0 (socket says 172.21.15.47:39288) 2024-05-11T12:33:18.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.888+0000 7f0a4ad9d700 1 -- 172.21.15.47:0/1957490780 learned_addr learned my addr 172.21.15.47:0/1957490780 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:18.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.889+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.890+0000 7f0a4ad9d700 1 -- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 msgr2=0x56533adb51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:18.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.890+0000 7f0a4ad9d700 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.890+0000 7f0a4ad9d700 1 -- 172.21.15.47:0/1957490780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 msgr2=0x56533adb56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:18.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.890+0000 7f0a4ad9d700 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:18.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.890+0000 7f0a4ad9d700 1 -- 172.21.15.47:0/1957490780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533ae28620 con 0x56533adaab50 2024-05-11T12:33:18.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.890+0000 7f0a4ad9d700 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adb5c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000ee80 tx=0x7f0a4000ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d856bf78ed1e018 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:18.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.891+0000 7f0a31ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:33:18.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.900+0000 7f0a31ffb700 1 -- 172.21.15.47:0/1957490780 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40024020 con 0x56533adaab50 2024-05-11T12:33:18.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.901+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a3c02d070 con 0x56533adaab50 2024-05-11T12:33:18.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.901+0000 7f0a31ffb700 1 -- 172.21.15.47:0/1957490780 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f0a4000f040 con 0x56533adaab50 2024-05-11T12:33:18.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.905+0000 7f0a31ffb700 1 -- 172.21.15.47:0/1957490780 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0a4002f050 con 0x56533adaab50 2024-05-11T12:33:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.905+0000 7f0a5b577b80 10 client.25411 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:18.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.905+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56533ae29980 con 0x56533adaab50 2024-05-11T12:33:18.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.909+0000 7f0a31ffb700 1 -- 172.21.15.47:0/1957490780 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a3c02d070 con 0x56533adaab50 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.910+0000 7f0a31ffb700 1 -- 172.21.15.47:0/1957490780 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0a40013450 con 0x56533adaab50 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.911+0000 7f0a5b577b80 10 client.25411 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.911+0000 7f0a5b577b80 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.911+0000 7f0a5b577b80 20 client.25411 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.911+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533ae2a090 0x56533ae2c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.911+0000 7f0a5b577b80 4 client.25411 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.911+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56533acff360 con 0x56533ae2a090 2024-05-11T12:33:18.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.913+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533ae2a090 0x56533ae2c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.923+0000 7f0a31ffb700 10 client.25411 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:18.923+0000 7f0a4a59c700 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533ae2a090 0x56533ae2c4b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda3dfe62cf47bd8 server_cookie=2128b1d5e8c44d86 in_seq=0 out_seq=0 2024-05-11T12:33:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.001+0000 7f0a31ffb700 1 -- 172.21.15.47:0/1957490780 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56533acff360 con 0x56533ae2a090 2024-05-11T12:33:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.001+0000 7f0a31ffb700 10 client.25411 handle_command_reply: tid=0 2024-05-11T12:33:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.001+0000 7f0a5b577b80 1 client.25411 shutdown 2024-05-11T12:33:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 msgr2=0x56533adb5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:19.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adb5c20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000ee80 tx=0x7f0a4000ef60 comp rx=0 tx=0).stop 2024-05-11T12:33:19.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 shutdown_connections 2024-05-11T12:33:19.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x56533ae2a090 0x56533ae2c4b0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x56533adaab50 0x56533adb5c20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x56533ada5120 0x56533adb51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 --2- 172.21.15.47:0/1957490780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56533ada5ae0 0x56533adb56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 >> 172.21.15.47:0/1957490780 conn(0x56533acd1ee0 msgr2=0x56533ada49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:19.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a31ffb700 0 client.25411 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 shutdown_connections 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 1 -- 172.21.15.47:0/1957490780 wait complete. 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.002+0000 7f0a5b577b80 20 client.25411 trim_cache size 0 max 16384 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "3c550218-522a-461a-9476-bfc4f36b9b3c", 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:33:19.004 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.003+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a340b2a90 msgr2=0x7f0a340b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.003+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a340b2a90 0x7f0a340b4f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0a40006fd0 tx=0x7f0a40009190 comp rx=0 tx=0).stop 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.003+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c111d30 msgr2=0x7f0a4c11c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.003+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c111d30 0x7f0a4c11c710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400ef10 tx=0x7f0a4400c3a0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 shutdown_connections 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a4c11cd30 0x7f0a4c1163a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f0a340b2a90 0x7f0a340b4f50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a4c10aa90 0x7f0a4c11c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 --2- 172.21.15.47:0/3539206591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0a4c111d30 0x7f0a4c11c710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 >> 172.21.15.47:0/3539206591 conn(0x7f0a4c06e6c0 msgr2=0x7f0a4c11e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 shutdown_connections 2024-05-11T12:33:19.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:19.004+0000 7f0a50ce1700 1 -- 172.21.15.47:0/3539206591 wait complete. 2024-05-11T12:33:19.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:19 smithi050 ceph-mon[34072]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.83k op/s 2024-05-11T12:33:19.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:19 smithi047 ceph-mon[30375]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.83k op/s 2024-05-11T12:33:19.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:19 smithi064 ceph-mon[34537]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 852 MiB/s wr, 1.83k op/s 2024-05-11T12:33:20.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:20 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:33:20.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:20 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:20.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:20 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:33:20.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:20 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:20.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:20 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:33:20.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:20 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:20.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:20 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:33:20.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:20 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:20 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:33:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:20 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:20 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:33:20.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:20 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:21.276 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:33:21.778 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:21 smithi047 ceph-mon[30375]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 792 MiB/s wr, 1.89k op/s 2024-05-11T12:33:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:21 smithi064 ceph-mon[34537]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 792 MiB/s wr, 1.89k op/s 2024-05-11T12:33:21.950 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:21 smithi050 ceph-mon[34072]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 792 MiB/s wr, 1.89k op/s 2024-05-11T12:33:22.308 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:33:23.598 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:23 smithi047 ceph-mon[30375]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 805 MiB/s wr, 1.74k op/s 2024-05-11T12:33:23.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:23 smithi064 ceph-mon[34537]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 805 MiB/s wr, 1.74k op/s 2024-05-11T12:33:23.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:23 smithi050 ceph-mon[34072]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 805 MiB/s wr, 1.74k op/s 2024-05-11T12:33:24.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:24 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:33:24.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:24 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:33:24.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:24 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:33:25.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:25 smithi064 ceph-mon[34537]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 734 MiB/s wr, 1.41k op/s 2024-05-11T12:33:25.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:25 smithi050 ceph-mon[34072]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 734 MiB/s wr, 1.41k op/s 2024-05-11T12:33:26.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:25 smithi047 ceph-mon[30375]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 734 MiB/s wr, 1.41k op/s 2024-05-11T12:33:27.822 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:27 smithi047 ceph-mon[30375]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 802 MiB/s wr, 1.53k op/s 2024-05-11T12:33:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:27 smithi064 ceph-mon[34537]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 802 MiB/s wr, 1.53k op/s 2024-05-11T12:33:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:27 smithi050 ceph-mon[34072]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 802 MiB/s wr, 1.53k op/s 2024-05-11T12:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.832+0000 7f631a636700 1 -- 172.21.15.47:0/1800005779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6314111d30 msgr2=0x7f631410a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.834+0000 7f631a636700 1 --2- 172.21.15.47:0/1800005779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6314111d30 0x7f631410a550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6304009990 tx=0x7f6304009ca0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.834+0000 7f631a636700 1 -- 172.21.15.47:0/1800005779 shutdown_connections 2024-05-11T12:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.834+0000 7f631a636700 1 --2- 172.21.15.47:0/1800005779 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f631411cd30 0x7f631411f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.834+0000 7f631a636700 1 --2- 172.21.15.47:0/1800005779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6314111d30 0x7f631410a550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.835+0000 7f631a636700 1 --2- 172.21.15.47:0/1800005779 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 0x7f631410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.835+0000 7f631a636700 1 -- 172.21.15.47:0/1800005779 >> 172.21.15.47:0/1800005779 conn(0x7f631406e6c0 msgr2=0x7f631406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.846+0000 7f631a636700 1 -- 172.21.15.47:0/1800005779 shutdown_connections 2024-05-11T12:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.846+0000 7f631a636700 1 -- 172.21.15.47:0/1800005779 wait complete. 2024-05-11T12:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.848+0000 7f631a636700 1 Processor -- start 2024-05-11T12:33:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.849+0000 7f631a636700 1 -- start start 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.855+0000 7f631a636700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 0x7f63141ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.856+0000 7f631a636700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6314111d30 0x7f63141aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.856+0000 7f6313fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 0x7f63141ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.856+0000 7f6313fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 0x7f63141ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:43570/0 (socket says 172.21.15.47:43570) 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.856+0000 7f6313fff700 1 -- 172.21.15.47:0/2325737959 learned_addr learned my addr 172.21.15.47:0/2325737959 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.856+0000 7f631a636700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631411cd30 0x7f63141a8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.857+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6314122640 con 0x7f6314111d30 2024-05-11T12:33:28.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.857+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f63141224c0 con 0x7f631411cd30 2024-05-11T12:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.857+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f63141227c0 con 0x7f631410aa90 2024-05-11T12:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.857+0000 7f6318bd3700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631411cd30 0x7f63141a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.858+0000 7f63137fe700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6314111d30 0x7f63141aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.858+0000 7f6313fff700 1 -- 172.21.15.47:0/2325737959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631411cd30 msgr2=0x7f63141a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.858+0000 7f6313fff700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631411cd30 0x7f63141a8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.858+0000 7f6313fff700 1 -- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6314111d30 msgr2=0x7f63141aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.858+0000 7f6313fff700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6314111d30 0x7f63141aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.858+0000 7f6313fff700 1 -- 172.21.15.47:0/2325737959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63141a9080 con 0x7f631410aa90 2024-05-11T12:33:28.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.859+0000 7f6313fff700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 0x7f63141ae760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63040098e0 tx=0x7f63040057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.860+0000 7f63117fa700 1 -- 172.21.15.47:0/2325737959 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400fa90 con 0x7f631410aa90 2024-05-11T12:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.860+0000 7f63117fa700 1 -- 172.21.15.47:0/2325737959 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f630400fc30 con 0x7f631410aa90 2024-05-11T12:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.860+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63141a9310 con 0x7f631410aa90 2024-05-11T12:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.860+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63141a9730 con 0x7f631410aa90 2024-05-11T12:33:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.861+0000 7f63117fa700 1 -- 172.21.15.47:0/2325737959 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400e640 con 0x7f631410aa90 2024-05-11T12:33:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.866+0000 7f63117fa700 1 -- 172.21.15.47:0/2325737959 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6304009d10 con 0x7f631410aa90 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.876+0000 7f6324eccb80 1 Processor -- start 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.880+0000 7f63117fa700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f62fc0b2a90 0x7f62fc0b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.880+0000 7f6324eccb80 1 -- start start 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.880+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bd7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.880+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd76a0 0x559c39bd2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.881+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bd2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.881+0000 7f6324eccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559c39b24940 con 0x559c39bd76a0 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.881+0000 7f6324eccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559c39a89da0 con 0x559c39bd2b30 2024-05-11T12:33:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.881+0000 7f6324eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559c39b20900 con 0x559c39bd6d80 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.881+0000 7f6318bd3700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f62fc0b2a90 0x7f62fc0b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.882+0000 7f63117fa700 1 -- 172.21.15.47:0/2325737959 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f63040e3300 con 0x7f631410aa90 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.882+0000 7f63137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd76a0 0x559c39bd2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.882+0000 7f63137fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd76a0 0x559c39bd2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40420/0 (socket says 172.21.15.47:40420) 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.882+0000 7f63137fe700 1 -- 172.21.15.47:0/3843014386 learned_addr learned my addr 172.21.15.47:0/3843014386 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.883+0000 7f63137fe700 1 -- 172.21.15.47:0/3843014386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 msgr2=0x559c39bd7160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.883+0000 7f63137fe700 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bd7160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.883+0000 7f63137fe700 1 -- 172.21.15.47:0/3843014386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 msgr2=0x559c39bd2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:28.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.883+0000 7f6318bd3700 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bd2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.883+0000 7f63137fe700 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bd2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.883+0000 7f63137fe700 1 -- 172.21.15.47:0/3843014386 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c39a718d0 con 0x559c39bd76a0 2024-05-11T12:33:28.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.886+0000 7f6318bd3700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f62fc0b2a90 0x7f62fc0b4f50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f631406ccc0 tx=0x7f630c00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:28.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.892+0000 7f63137fe700 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd76a0 0x559c39bd2510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6308009d20 tx=0x7f6308011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b257a02a669644 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.895+0000 7f62fa7fc700 1 -- 172.21.15.47:0/3843014386 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630801e070 con 0x559c39bd76a0 2024-05-11T12:33:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.895+0000 7f62fa7fc700 1 -- 172.21.15.47:0/3843014386 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6308019ae0 con 0x559c39bd76a0 2024-05-11T12:33:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.896+0000 7f6324eccb80 1 -- 172.21.15.47:0/3843014386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd76a0 msgr2=0x559c39bd2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.896+0000 7f6324eccb80 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd76a0 0x559c39bd2510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6308009d20 tx=0x7f6308011720 comp rx=0 tx=0).stop 2024-05-11T12:33:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.897+0000 7f6324eccb80 1 -- 172.21.15.47:0/3843014386 shutdown_connections 2024-05-11T12:33:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.897+0000 7f6324eccb80 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bd2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.897+0000 7f6324eccb80 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bd7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.898+0000 7f6324eccb80 1 --2- 172.21.15.47:0/3843014386 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd76a0 0x559c39bd2510 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.898+0000 7f6324eccb80 1 -- 172.21.15.47:0/3843014386 >> 172.21.15.47:0/3843014386 conn(0x559c39afeb70 msgr2=0x559c39bccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:28.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.898+0000 7f6324eccb80 1 -- 172.21.15.47:0/3843014386 shutdown_connections 2024-05-11T12:33:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.898+0000 7f6324eccb80 1 -- 172.21.15.47:0/3843014386 wait complete. 2024-05-11T12:33:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.899+0000 7f6324eccb80 1 -- start start 2024-05-11T12:33:28.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.900+0000 7f6324eccb80 1 Processor -- start 2024-05-11T12:33:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.901+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bde490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.901+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd6d80 0x559c39bdb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.901+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd76a0 0x559c39bdbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.901+0000 7f6324eccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559c39bdf2c0 con 0x559c39bd6d80 2024-05-11T12:33:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.902+0000 7f6324eccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559c39added0 con 0x559c39bd2b30 2024-05-11T12:33:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.902+0000 7f6324eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559c39bdf440 con 0x559c39bd76a0 2024-05-11T12:33:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.902+0000 7f6313fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bde490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.902+0000 7f6324eccb80 10 client.0 _ll_register_callbacks cb 0x559c39abc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6313fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bde490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:38074/0 (socket says 172.21.15.47:38074) 2024-05-11T12:33:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6313fff700 1 -- 172.21.15.47:0/95435330 learned_addr learned my addr 172.21.15.47:0/95435330 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6318bd3700 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd76a0 0x559c39bdbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f63137fe700 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd6d80 0x559c39bdb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6313fff700 1 -- 172.21.15.47:0/95435330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd76a0 msgr2=0x559c39bdbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6313fff700 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd76a0 0x559c39bdbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6313fff700 1 -- 172.21.15.47:0/95435330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd6d80 msgr2=0x559c39bdb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6313fff700 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd6d80 0x559c39bdb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.903+0000 7f6313fff700 1 -- 172.21.15.47:0/95435330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c39adb760 con 0x559c39bd2b30 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.904+0000 7f6313fff700 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bde490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f63040a8a60 tx=0x7f63040b34c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acd690bb42184588 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.904+0000 7f62f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.904+0000 7f62f37fe700 1 -- 172.21.15.47:0/95435330 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040b1060 con 0x559c39bd2b30 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.904+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6308019ae0 con 0x559c39bd2b30 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.905+0000 7f62f37fe700 1 -- 172.21.15.47:0/95435330 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f63040050e0 con 0x559c39bd2b30 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.905+0000 7f62f37fe700 1 -- 172.21.15.47:0/95435330 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63040aaf60 con 0x559c39bd2b30 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.905+0000 7f6324eccb80 10 client.25385 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.905+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559c39be9990 con 0x559c39bd2b30 2024-05-11T12:33:28.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.905+0000 7f62f37fe700 1 -- 172.21.15.47:0/95435330 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63040095c0 con 0x559c39bd2b30 2024-05-11T12:33:28.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.906+0000 7f62f37fe700 1 -- 172.21.15.47:0/95435330 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f63040a7210 con 0x559c39bd2b30 2024-05-11T12:33:28.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.906+0000 7f6324eccb80 10 client.25385 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:28.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.906+0000 7f6324eccb80 10 client.25385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:28.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.906+0000 7f6324eccb80 20 client.25385 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:28.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.906+0000 7f6324eccb80 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39bebf80 0x559c39bee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.906+0000 7f6324eccb80 4 client.25385 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:33:28.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.906+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6308019ae0 con 0x559c39bebf80 2024-05-11T12:33:28.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.907+0000 7f63137fe700 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39bebf80 0x559c39bee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.911+0000 7f63137fe700 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39bebf80 0x559c39bee3a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c6e662216417c8 server_cookie=1c5e77280a484850 in_seq=0 out_seq=0 2024-05-11T12:33:28.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.911+0000 7f62f37fe700 10 client.25385 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:28.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.914+0000 7f62f37fe700 1 -- 172.21.15.47:0/95435330 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f6308019ae0 con 0x559c39bebf80 2024-05-11T12:33:28.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.914+0000 7f62f37fe700 10 client.25385 handle_command_reply: tid=0 2024-05-11T12:33:28.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.914+0000 7f6324eccb80 1 client.25385 shutdown 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.921+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 msgr2=0x559c39bde490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.921+0000 7f6324eccb80 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bde490 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f63040a8a60 tx=0x7f63040b34c0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.921+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 shutdown_connections 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.921+0000 7f6324eccb80 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39bebf80 0x559c39bee3a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.922+0000 7f6324eccb80 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd76a0 0x559c39bdbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.922+0000 7f6324eccb80 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd2b30 0x559c39bde490 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.922+0000 7f6324eccb80 1 --2- 172.21.15.47:0/95435330 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd6d80 0x559c39bdb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.922+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 >> 172.21.15.47:0/95435330 conn(0x559c39afeb70 msgr2=0x559c39bd83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.922+0000 7f62f37fe700 0 client.25385 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.922+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 shutdown_connections 2024-05-11T12:33:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.922+0000 7f6324eccb80 1 -- 172.21.15.47:0/95435330 wait complete. 2024-05-11T12:33:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.923+0000 7f6324eccb80 20 client.25385 trim_cache size 0 max 16384 2024-05-11T12:33:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.971+0000 7f6324eccb80 1 Processor -- start 2024-05-11T12:33:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.972+0000 7f6324eccb80 1 -- start start 2024-05-11T12:33:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.972+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bdbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.972+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bc7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.972+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 0x559c39bd1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.972+0000 7f6324eccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f630000b7e0 con 0x559c39bd2b30 2024-05-11T12:33:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.972+0000 7f6324eccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f630000e060 con 0x559c39bd76a0 2024-05-11T12:33:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.972+0000 7f6324eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f630000be50 con 0x559c39bd6d80 2024-05-11T12:33:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bdbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bdbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40436/0 (socket says 172.21.15.47:40436) 2024-05-11T12:33:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 -- 172.21.15.47:0/1015741343 learned_addr learned my addr 172.21.15.47:0/1015741343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 -- 172.21.15.47:0/1015741343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 msgr2=0x559c39bc7940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:33:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 --2- 172.21.15.47:0/1015741343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bc7940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 -- 172.21.15.47:0/1015741343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 msgr2=0x559c39bd1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 --2- 172.21.15.47:0/1015741343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 0x559c39bd1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.973+0000 7f6313fff700 1 -- 172.21.15.47:0/1015741343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c39bdf4d0 con 0x559c39bd2b30 2024-05-11T12:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.974+0000 7f6313fff700 1 --2- 172.21.15.47:0/1015741343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bdbda0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f63040a7d90 tx=0x7f630400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed35ec5529899506 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.974+0000 7f62f9ffb700 1 -- 172.21.15.47:0/1015741343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040b1200 con 0x559c39bd2b30 2024-05-11T12:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.974+0000 7f62f9ffb700 1 -- 172.21.15.47:0/1015741343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63040e0ba0 con 0x559c39bd2b30 2024-05-11T12:33:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.974+0000 7f62f9ffb700 1 -- 172.21.15.47:0/1015741343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400fde0 con 0x559c39bd2b30 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 -- 172.21.15.47:0/1015741343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 msgr2=0x559c39bdbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 --2- 172.21.15.47:0/1015741343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bdbda0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f63040a7d90 tx=0x7f630400b610 comp rx=0 tx=0).stop 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 -- 172.21.15.47:0/1015741343 shutdown_connections 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 --2- 172.21.15.47:0/1015741343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 0x559c39bd1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 --2- 172.21.15.47:0/1015741343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bdbda0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 --2- 172.21.15.47:0/1015741343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bc7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 -- 172.21.15.47:0/1015741343 >> 172.21.15.47:0/1015741343 conn(0x559c39afeb70 msgr2=0x559c39bd1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 -- 172.21.15.47:0/1015741343 shutdown_connections 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.976+0000 7f6324eccb80 1 -- 172.21.15.47:0/1015741343 wait complete. 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.977+0000 7f6324eccb80 1 -- start start 2024-05-11T12:33:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.977+0000 7f6324eccb80 1 Processor -- start 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bf39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bf3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6324eccb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 0x559c39bf0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6324eccb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f630000c520 con 0x559c39bd2b30 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6324eccb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f630000d990 con 0x559c39bd76a0 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6324eccb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f630000cbf0 con 0x559c39bd6d80 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6324eccb80 10 client.0 _ll_register_callbacks cb 0x559c39abc950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6313fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bf39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6313fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bf39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40450/0 (socket says 172.21.15.47:40450) 2024-05-11T12:33:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.978+0000 7f6313fff700 1 -- 172.21.15.47:0/500621634 learned_addr learned my addr 172.21.15.47:0/500621634 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:28.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.979+0000 7f6313fff700 1 -- 172.21.15.47:0/500621634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 msgr2=0x559c39bf3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:28.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.979+0000 7f6313fff700 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bf3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.979+0000 7f6313fff700 1 -- 172.21.15.47:0/500621634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 msgr2=0x559c39bf0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.979+0000 7f6313fff700 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 0x559c39bf0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.979+0000 7f6313fff700 1 -- 172.21.15.47:0/500621634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c39bf0ed0 con 0x559c39bd2b30 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.979+0000 7f6313fff700 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bf39c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f63040cbac0 tx=0x7f63040a74c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21753760a7390903 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.979+0000 7f62f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.980+0000 7f62f37fe700 1 -- 172.21.15.47:0/500621634 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63040aad30 con 0x559c39bd2b30 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.980+0000 7f62f37fe700 1 -- 172.21.15.47:0/500621634 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f63040a7de0 con 0x559c39bd2b30 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.980+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63040e0ba0 con 0x559c39bd2b30 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.980+0000 7f62f37fe700 1 -- 172.21.15.47:0/500621634 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63040ab300 con 0x559c39bd2b30 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.980+0000 7f6324eccb80 10 client.15813 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.980+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559c39bf0ed0 con 0x559c39bd2b30 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f62f37fe700 1 -- 172.21.15.47:0/500621634 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63040e0ba0 con 0x559c39bd2b30 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f62f37fe700 1 -- 172.21.15.47:0/500621634 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f63040cb3b0 con 0x559c39bd2b30 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f6324eccb80 10 client.15813 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f6324eccb80 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f6324eccb80 20 client.15813 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f6324eccb80 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39c57060 0x559c39c59450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f6324eccb80 4 client.15813 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:33:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.981+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c39b32320 con 0x559c39c57060 2024-05-11T12:33:28.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.983+0000 7f63137fe700 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39c57060 0x559c39c59450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.985+0000 7f62f37fe700 10 client.15813 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.985+0000 7f63137fe700 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39c57060 0x559c39c59450 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df851fa7bbe355 server_cookie=3785460ebf60a996 in_seq=0 out_seq=0 2024-05-11T12:33:28.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.989+0000 7f62f37fe700 1 -- 172.21.15.47:0/500621634 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559c39b32320 con 0x559c39c57060 2024-05-11T12:33:28.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.989+0000 7f62f37fe700 10 client.15813 handle_command_reply: tid=0 2024-05-11T12:33:28.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.989+0000 7f6324eccb80 1 client.15813 shutdown 2024-05-11T12:33:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 msgr2=0x559c39bf39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bf39c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f63040cbac0 tx=0x7f63040a74c0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 shutdown_connections 2024-05-11T12:33:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559c39c57060 0x559c39c59450 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559c39bd76a0 0x559c39bf0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559c39bd2b30 0x559c39bf39c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 --2- 172.21.15.47:0/500621634 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559c39bd6d80 0x559c39bf3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.992+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 >> 172.21.15.47:0/500621634 conn(0x559c39afeb70 msgr2=0x559c39be8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:28.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.997+0000 7f62f37fe700 0 client.15813 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:29.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.998+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 shutdown_connections 2024-05-11T12:33:29.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:28.998+0000 7f6324eccb80 1 -- 172.21.15.47:0/500621634 wait complete. 2024-05-11T12:33:29.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.003+0000 7f6324eccb80 20 client.15813 trim_cache size 0 max 16384 2024-05-11T12:33:29.011 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:33:29.011 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:33:29.011 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:33:29.012 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:33:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.009+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f62fc0b2a90 msgr2=0x7f62fc0b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.009+0000 7f631a636700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f62fc0b2a90 0x7f62fc0b4f50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f631406ccc0 tx=0x7f630c00c040 comp rx=0 tx=0).stop 2024-05-11T12:33:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.009+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 msgr2=0x7f63141ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.009+0000 7f631a636700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 0x7f63141ae760 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63040098e0 tx=0x7f63040057c0 comp rx=0 tx=0).stop 2024-05-11T12:33:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.014+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 shutdown_connections 2024-05-11T12:33:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.014+0000 7f631a636700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f62fc0b2a90 0x7f62fc0b4f50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.014+0000 7f631a636700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f631411cd30 0x7f63141a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.014+0000 7f631a636700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f631410aa90 0x7f63141ae760 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.014+0000 7f631a636700 1 --2- 172.21.15.47:0/2325737959 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6314111d30 0x7f63141aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.014+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 >> 172.21.15.47:0/2325737959 conn(0x7f631406e6c0 msgr2=0x7f631411ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.019+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 shutdown_connections 2024-05-11T12:33:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:29.019+0000 7f631a636700 1 -- 172.21.15.47:0/2325737959 wait complete. 2024-05-11T12:33:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:29 smithi064 ceph-mon[34537]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 885 MiB/s wr, 1.87k op/s 2024-05-11T12:33:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:29 smithi050 ceph-mon[34072]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 885 MiB/s wr, 1.87k op/s 2024-05-11T12:33:30.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:29 smithi047 ceph-mon[30375]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 885 MiB/s wr, 1.87k op/s 2024-05-11T12:33:31.311 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:33:31.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:33:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:31 smithi050 ceph-mon[34072]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 815 MiB/s wr, 1.76k op/s 2024-05-11T12:33:32.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:31 smithi047 ceph-mon[30375]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 815 MiB/s wr, 1.76k op/s 2024-05-11T12:33:32.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:31 smithi064 ceph-mon[34537]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 815 MiB/s wr, 1.76k op/s 2024-05-11T12:33:32.307 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:33:33.895 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:33 smithi047 ceph-mon[30375]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.89k op/s 2024-05-11T12:33:33.916 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:33 smithi064 ceph-mon[34537]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.89k op/s 2024-05-11T12:33:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:33 smithi050 ceph-mon[34072]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 846 MiB/s wr, 1.89k op/s 2024-05-11T12:33:35.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:35 smithi050 ceph-mon[34072]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 737 MiB/s wr, 1.77k op/s 2024-05-11T12:33:36.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:35 smithi047 ceph-mon[30375]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 737 MiB/s wr, 1.77k op/s 2024-05-11T12:33:36.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:35 smithi064 ceph-mon[34537]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 737 MiB/s wr, 1.77k op/s 2024-05-11T12:33:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.264+0000 7f1ebc806700 1 -- 172.21.15.47:0/302421307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb81023d0 msgr2=0x7f1eb8102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.264+0000 7f1ebc806700 1 --2- 172.21.15.47:0/302421307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb81023d0 0x7f1eb8102850 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1eb001c0d0 tx=0x7f1eb001c3e0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.265+0000 7f1ebc806700 1 -- 172.21.15.47:0/302421307 shutdown_connections 2024-05-11T12:33:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.265+0000 7f1ebc806700 1 --2- 172.21.15.47:0/302421307 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8102d90 0x7f1eb810f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.265+0000 7f1ebc806700 1 --2- 172.21.15.47:0/302421307 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb8108460 0x7f1eb8108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.265+0000 7f1ebc806700 1 --2- 172.21.15.47:0/302421307 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb81023d0 0x7f1eb8102850 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.265+0000 7f1ebc806700 1 -- 172.21.15.47:0/302421307 >> 172.21.15.47:0/302421307 conn(0x7f1eb80fdee0 msgr2=0x7f1eb8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:37.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.267+0000 7f1ebc806700 1 -- 172.21.15.47:0/302421307 shutdown_connections 2024-05-11T12:33:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.272+0000 7f1ebc806700 1 -- 172.21.15.47:0/302421307 wait complete. 2024-05-11T12:33:37.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.273+0000 7f1ebc806700 1 Processor -- start 2024-05-11T12:33:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.276+0000 7f1ebc806700 1 -- start start 2024-05-11T12:33:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1ebc806700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb81023d0 0x7f1eb8194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:37.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1ebc806700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb8102d90 0x7f1eb81950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1ebc806700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8108460 0x7f1eb8195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1ebc806700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb806f830 con 0x7f1eb8102d90 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1ebc806700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb806f6b0 con 0x7f1eb8108460 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1ebc806700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb806f9b0 con 0x7f1eb81023d0 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8108460 0x7f1eb8195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1eb7fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8108460 0x7f1eb8195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:59378/0 (socket says 172.21.15.47:59378) 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.277+0000 7f1eb7fff700 1 -- 172.21.15.47:0/770458595 learned_addr learned my addr 172.21.15.47:0/770458595 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.278+0000 7f1eb7fff700 1 -- 172.21.15.47:0/770458595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb81023d0 msgr2=0x7f1eb8194ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.278+0000 7f1eb7fff700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb81023d0 0x7f1eb8194ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.278+0000 7f1eb7fff700 1 -- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb8102d90 msgr2=0x7f1eb81950e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.278+0000 7f1eb7fff700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb8102d90 0x7f1eb81950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.278+0000 7f1eb7fff700 1 -- 172.21.15.47:0/770458595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb810de10 con 0x7f1eb8108460 2024-05-11T12:33:37.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.281+0000 7f1eb7fff700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8108460 0x7f1eb8195620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b4a0 tx=0x7f1ea800b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:37.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.282+0000 7f1eb4ff9700 1 -- 172.21.15.47:0/770458595 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8015020 con 0x7f1eb8108460 2024-05-11T12:33:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.282+0000 7f1eb4ff9700 1 -- 172.21.15.47:0/770458595 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea8011ce0 con 0x7f1eb8108460 2024-05-11T12:33:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.282+0000 7f1eb4ff9700 1 -- 172.21.15.47:0/770458595 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8010e10 con 0x7f1eb8108460 2024-05-11T12:33:37.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.283+0000 7f1ebc806700 1 -- 172.21.15.47:0/770458595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb810e100 con 0x7f1eb8108460 2024-05-11T12:33:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.283+0000 7f1ebc806700 1 -- 172.21.15.47:0/770458595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb810e530 con 0x7f1eb8108460 2024-05-11T12:33:37.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.285+0000 7f1ebc806700 1 -- 172.21.15.47:0/770458595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb8103b10 con 0x7f1eb8108460 2024-05-11T12:33:37.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.293+0000 7f1eb4ff9700 1 -- 172.21.15.47:0/770458595 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ea8018020 con 0x7f1eb8108460 2024-05-11T12:33:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.296+0000 7f1eb4ff9700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ea00b2a90 0x7f1ea00b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.297+0000 7f1eb4ff9700 1 -- 172.21.15.47:0/770458595 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1ea80e20b0 con 0x7f1eb8108460 2024-05-11T12:33:37.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.297+0000 7f1eb4ff9700 1 -- 172.21.15.47:0/770458595 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ea801f020 con 0x7f1eb8108460 2024-05-11T12:33:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.333+0000 7f1eb77fe700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ea00b2a90 0x7f1ea00b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:37.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.337+0000 7f1eb77fe700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ea00b2a90 0x7f1ea00b4f50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009710 tx=0x7f1eac006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:37.939 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:37 smithi047 ceph-mon[30375]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 MiB/s wr, 2.11k op/s 2024-05-11T12:33:37.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.937+0000 7f1ebc806700 1 -- 172.21.15.47:0/770458595 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1eb8071210 con 0x7f1eb8108460 2024-05-11T12:33:37.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.940+0000 7f1eb4ff9700 1 -- 172.21.15.47:0/770458595 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f1ea80a9d20 con 0x7f1eb8108460 2024-05-11T12:33:37.942 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:33:37.943 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:33:37.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:37 smithi050 ceph-mon[34072]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 MiB/s wr, 2.11k op/s 2024-05-11T12:33:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.949+0000 7f1e9e7fc700 1 -- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ea00b2a90 msgr2=0x7f1ea00b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.949+0000 7f1e9e7fc700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ea00b2a90 0x7f1ea00b4f50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009710 tx=0x7f1eac006c60 comp rx=0 tx=0).stop 2024-05-11T12:33:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.949+0000 7f1e9e7fc700 1 -- 172.21.15.47:0/770458595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8108460 msgr2=0x7f1eb8195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.949+0000 7f1e9e7fc700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8108460 0x7f1eb8195620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800b4a0 tx=0x7f1ea800b860 comp rx=0 tx=0).stop 2024-05-11T12:33:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.956+0000 7f1e9e7fc700 1 -- 172.21.15.47:0/770458595 shutdown_connections 2024-05-11T12:33:37.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.956+0000 7f1e9e7fc700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ea00b2a90 0x7f1ea00b4f50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.956+0000 7f1e9e7fc700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1eb8108460 0x7f1eb8195620 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.956+0000 7f1e9e7fc700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1eb81023d0 0x7f1eb8194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.956+0000 7f1e9e7fc700 1 --2- 172.21.15.47:0/770458595 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1eb8102d90 0x7f1eb81950e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.956+0000 7f1e9e7fc700 1 -- 172.21.15.47:0/770458595 >> 172.21.15.47:0/770458595 conn(0x7f1eb80fdee0 msgr2=0x7f1eb80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.956+0000 7f1e9e7fc700 1 -- 172.21.15.47:0/770458595 shutdown_connections 2024-05-11T12:33:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:37.957+0000 7f1e9e7fc700 1 -- 172.21.15.47:0/770458595 wait complete. 2024-05-11T12:33:37.975 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:33:38.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:37 smithi064 ceph-mon[34537]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 840 MiB/s wr, 2.11k op/s 2024-05-11T12:33:38.848 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:38 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/770458595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:33:39.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:38 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/770458595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:33:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:38 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/770458595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:33:40.034 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:39 smithi047 ceph-mon[30375]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 2.02k op/s 2024-05-11T12:33:40.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:39 smithi064 ceph-mon[34537]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 2.02k op/s 2024-05-11T12:33:40.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:39 smithi050 ceph-mon[34072]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 832 MiB/s wr, 2.02k op/s 2024-05-11T12:33:40.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:33:41.275 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:33:41.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:41 smithi047 ceph-mon[30375]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 802 MiB/s wr, 1.82k op/s 2024-05-11T12:33:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:41 smithi064 ceph-mon[34537]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 802 MiB/s wr, 1.82k op/s 2024-05-11T12:33:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:41 smithi050 ceph-mon[34072]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 802 MiB/s wr, 1.82k op/s 2024-05-11T12:33:44.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:43 smithi047 ceph-mon[30375]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 816 MiB/s wr, 1.80k op/s 2024-05-11T12:33:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:43 smithi064 ceph-mon[34537]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 816 MiB/s wr, 1.80k op/s 2024-05-11T12:33:44.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:43 smithi050 ceph-mon[34072]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 816 MiB/s wr, 1.80k op/s 2024-05-11T12:33:46.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:45 smithi047 ceph-mon[30375]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 737 MiB/s wr, 1.46k op/s 2024-05-11T12:33:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:45 smithi064 ceph-mon[34537]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 737 MiB/s wr, 1.46k op/s 2024-05-11T12:33:46.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:45 smithi050 ceph-mon[34072]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 737 MiB/s wr, 1.46k op/s 2024-05-11T12:33:47.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.041+0000 7faa4d654700 1 -- 172.21.15.47:0/541660985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa40094ee0 msgr2=0x7faa40095360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.042+0000 7faa4d654700 1 --2- 172.21.15.47:0/541660985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa40094ee0 0x7faa40095360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faa340099c0 tx=0x7faa34009cd0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.052+0000 7faa4d654700 1 -- 172.21.15.47:0/541660985 shutdown_connections 2024-05-11T12:33:47.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.052+0000 7faa4d654700 1 --2- 172.21.15.47:0/541660985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa400958a0 0x7faa400a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.052+0000 7faa4d654700 1 --2- 172.21.15.47:0/541660985 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa4009aca0 0x7faa4009b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.052+0000 7faa4d654700 1 --2- 172.21.15.47:0/541660985 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa40094ee0 0x7faa40095360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.052+0000 7faa4d654700 1 -- 172.21.15.47:0/541660985 >> 172.21.15.47:0/541660985 conn(0x7faa4000a7a0 msgr2=0x7faa4000abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:47.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.053+0000 7faa4d654700 1 -- 172.21.15.47:0/541660985 shutdown_connections 2024-05-11T12:33:47.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.053+0000 7faa4d654700 1 -- 172.21.15.47:0/541660985 wait complete. 2024-05-11T12:33:47.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.054+0000 7faa4d654700 1 Processor -- start 2024-05-11T12:33:47.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.055+0000 7faa4d654700 1 -- start start 2024-05-11T12:33:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.056+0000 7faa4d654700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa40094ee0 0x7faa4012b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.056+0000 7faa4d654700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa400958a0 0x7faa4012b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.056+0000 7faa4d654700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa4009aca0 0x7faa4012fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.056+0000 7faa4d654700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa400a4f30 con 0x7faa4009aca0 2024-05-11T12:33:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.056+0000 7faa4d654700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa400a4db0 con 0x7faa400958a0 2024-05-11T12:33:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.056+0000 7faa4d654700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa400a50b0 con 0x7faa40094ee0 2024-05-11T12:33:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.057+0000 7faa4ce53700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa4009aca0 0x7faa4012fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.057+0000 7faa4ce53700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa4009aca0 0x7faa4012fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33130/0 (socket says 172.21.15.47:33130) 2024-05-11T12:33:47.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.057+0000 7faa4ce53700 1 -- 172.21.15.47:0/849976192 learned_addr learned my addr 172.21.15.47:0/849976192 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:47.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.058+0000 7faa477fe700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa400958a0 0x7faa4012b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.059+0000 7faa47fff700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa40094ee0 0x7faa4012b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.060+0000 7faa4ce53700 1 -- 172.21.15.47:0/849976192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa40094ee0 msgr2=0x7faa4012b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.060+0000 7faa4ce53700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa40094ee0 0x7faa4012b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.060+0000 7faa4ce53700 1 -- 172.21.15.47:0/849976192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa400958a0 msgr2=0x7faa4012b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.060+0000 7faa4ce53700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa400958a0 0x7faa4012b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.060+0000 7faa4ce53700 1 -- 172.21.15.47:0/849976192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa40130450 con 0x7faa4009aca0 2024-05-11T12:33:47.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.060+0000 7faa47fff700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa40094ee0 0x7faa4012b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:33:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.062+0000 7faa4ce53700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa4009aca0 0x7faa4012fd10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa48052c90 tx=0x7faa48075c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:47.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.062+0000 7faa457fa700 1 -- 172.21.15.47:0/849976192 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4806cc40 con 0x7faa4009aca0 2024-05-11T12:33:47.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.063+0000 7faa457fa700 1 -- 172.21.15.47:0/849976192 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa4806c0d0 con 0x7faa4009aca0 2024-05-11T12:33:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.068+0000 7faa457fa700 1 -- 172.21.15.47:0/849976192 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4807c460 con 0x7faa4009aca0 2024-05-11T12:33:47.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.070+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa40130740 con 0x7faa4009aca0 2024-05-11T12:33:47.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.070+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa40004d40 con 0x7faa4009aca0 2024-05-11T12:33:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.080+0000 7faa457fa700 1 -- 172.21.15.47:0/849976192 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7faa4806a4d0 con 0x7faa4009aca0 2024-05-11T12:33:47.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.082+0000 7faa5914cb80 1 Processor -- start 2024-05-11T12:33:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.083+0000 7faa5914cb80 1 -- start start 2024-05-11T12:33:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.084+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4c780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.084+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7c000 0x558be4c785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.084+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c8cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.084+0000 7faa5914cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558be4bca860 con 0x558be4c7c000 2024-05-11T12:33:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.084+0000 7faa5914cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558be4b697c0 con 0x558be4c7bc20 2024-05-11T12:33:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.084+0000 7faa5914cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558be4bc6820 con 0x558be4c78c00 2024-05-11T12:33:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.084+0000 7faa457fa700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa380b28e0 0x7faa380b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.085+0000 7faa457fa700 1 -- 172.21.15.47:0/849976192 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7faa48150ff0 con 0x7faa4009aca0 2024-05-11T12:33:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.085+0000 7faa47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7c000 0x558be4c785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.085+0000 7faa47fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7c000 0x558be4c785e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33142/0 (socket says 172.21.15.47:33142) 2024-05-11T12:33:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.085+0000 7faa47fff700 1 -- 172.21.15.47:0/644622598 learned_addr learned my addr 172.21.15.47:0/644622598 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:47.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.086+0000 7faa4ce53700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa380b28e0 0x7faa380b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.087+0000 7faa4ce53700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa380b28e0 0x7faa380b4da0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faa48115790 tx=0x7faa4807f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.087+0000 7faa47fff700 1 -- 172.21.15.47:0/644622598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 msgr2=0x558be4c8cd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:33:47.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.087+0000 7faa47fff700 1 --2- 172.21.15.47:0/644622598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c8cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.087+0000 7faa47fff700 1 -- 172.21.15.47:0/644622598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 msgr2=0x558be4c780a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:33:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.087+0000 7faa47fff700 1 --2- 172.21.15.47:0/644622598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4c780a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.087+0000 7faa47fff700 1 -- 172.21.15.47:0/644622598 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be4b178d0 con 0x558be4c7c000 2024-05-11T12:33:47.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.088+0000 7faa47fff700 1 --2- 172.21.15.47:0/644622598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7c000 0x558be4c785e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00aa50 tx=0x7faa3c00ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e696fdc76b8630d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:47.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.091+0000 7faa327fc700 1 -- 172.21.15.47:0/644622598 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c015070 con 0x558be4c7c000 2024-05-11T12:33:47.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.091+0000 7faa327fc700 1 -- 172.21.15.47:0/644622598 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c0113f0 con 0x558be4c7c000 2024-05-11T12:33:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.093+0000 7faa5914cb80 1 -- 172.21.15.47:0/644622598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7c000 msgr2=0x558be4c785e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.093+0000 7faa5914cb80 1 --2- 172.21.15.47:0/644622598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7c000 0x558be4c785e0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00aa50 tx=0x7faa3c00ad60 comp rx=0 tx=0).stop 2024-05-11T12:33:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.093+0000 7faa5914cb80 1 -- 172.21.15.47:0/644622598 shutdown_connections 2024-05-11T12:33:47.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.093+0000 7faa5914cb80 1 --2- 172.21.15.47:0/644622598 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c8cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.093+0000 7faa5914cb80 1 --2- 172.21.15.47:0/644622598 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4c780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.093+0000 7faa5914cb80 1 --2- 172.21.15.47:0/644622598 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7c000 0x558be4c785e0 secure :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00aa50 tx=0x7faa3c00ad60 comp rx=0 tx=0).stop 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.093+0000 7faa5914cb80 1 -- 172.21.15.47:0/644622598 >> 172.21.15.47:0/644622598 conn(0x558be4ba4e60 msgr2=0x558be4c72cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.094+0000 7faa5914cb80 1 -- 172.21.15.47:0/644622598 shutdown_connections 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.094+0000 7faa5914cb80 1 -- 172.21.15.47:0/644622598 wait complete. 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.095+0000 7faa5914cb80 1 -- start start 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.095+0000 7faa5914cb80 1 Processor -- start 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.096+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.096+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7bc20 0x558be4c89ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.096+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c8a3e0 0x558be4c8a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.096+0000 7faa5914cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558be4c8d820 con 0x558be4c7bc20 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.096+0000 7faa5914cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558be4b30c30 con 0x558be4c78c00 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.096+0000 7faa5914cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558be4c8d9a0 con 0x558be4c8a3e0 2024-05-11T12:33:47.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.096+0000 7faa5914cb80 10 client.0 _ll_register_callbacks cb 0x558be4b62920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.097+0000 7faa477fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.097+0000 7faa477fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c8cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:39352/0 (socket says 172.21.15.47:39352) 2024-05-11T12:33:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.097+0000 7faa477fe700 1 -- 172.21.15.47:0/71125086 learned_addr learned my addr 172.21.15.47:0/71125086 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:47.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.097+0000 7faa477fe700 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c8a3e0 0x558be4c8a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.102+0000 7faa47fff700 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7bc20 0x558be4c89ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.105+0000 7faa477fe700 1 -- 172.21.15.47:0/71125086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c8a3e0 msgr2=0x558be4c8a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:47.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.105+0000 7faa477fe700 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c8a3e0 0x558be4c8a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.105+0000 7faa477fe700 1 -- 172.21.15.47:0/71125086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7bc20 msgr2=0x558be4c89ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:47.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.105+0000 7faa477fe700 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7bc20 0x558be4c89ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.105+0000 7faa477fe700 1 -- 172.21.15.47:0/71125086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be4b66690 con 0x558be4c78c00 2024-05-11T12:33:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.107+0000 7faa477fe700 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c8cba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa3400b580 tx=0x7faa3401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25aaf9c1e13ae05e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:47.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.107+0000 7faa277fe700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:33:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.108+0000 7faa277fe700 1 -- 172.21.15.47:0/71125086 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34003960 con 0x558be4c78c00 2024-05-11T12:33:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.108+0000 7faa277fe700 1 -- 172.21.15.47:0/71125086 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7faa34003b00 con 0x558be4c78c00 2024-05-11T12:33:47.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.108+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa3c0113f0 con 0x558be4c78c00 2024-05-11T12:33:47.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.111+0000 7faa277fe700 1 -- 172.21.15.47:0/71125086 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa3402e050 con 0x558be4c78c00 2024-05-11T12:33:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.116+0000 7faa5914cb80 10 client.25397 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:47.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.116+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558be4c98720 con 0x558be4c78c00 2024-05-11T12:33:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.121+0000 7faa277fe700 1 -- 172.21.15.47:0/71125086 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c0113f0 con 0x558be4c78c00 2024-05-11T12:33:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.122+0000 7faa277fe700 1 -- 172.21.15.47:0/71125086 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7faa34027040 con 0x558be4c78c00 2024-05-11T12:33:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.125+0000 7faa5914cb80 10 client.25397 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.125+0000 7faa5914cb80 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.125+0000 7faa5914cb80 20 client.25397 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.125+0000 7faa5914cb80 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4c9ad60 0x558be4c9d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.125+0000 7faa5914cb80 4 client.25397 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:33:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.125+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be4bd8240 con 0x558be4c9ad60 2024-05-11T12:33:47.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.130+0000 7faa47fff700 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4c9ad60 0x558be4c9d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.154+0000 7faa47fff700 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4c9ad60 0x558be4c9d190 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b623ae20e86f8a8 server_cookie=5a6ec39cd21f45be in_seq=0 out_seq=0 2024-05-11T12:33:47.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.157+0000 7faa277fe700 10 client.25397 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:47.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.160+0000 7faa277fe700 1 -- 172.21.15.47:0/71125086 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558be4bd8240 con 0x558be4c9ad60 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.160+0000 7faa277fe700 10 client.25397 handle_command_reply: tid=0 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 client.25397 shutdown 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 msgr2=0x558be4c8cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c8cba0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa3400b580 tx=0x7faa3401a040 comp rx=0 tx=0).stop 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 shutdown_connections 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4c9ad60 0x558be4c9d190 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c8a3e0 0x558be4c8a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c8cba0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 --2- 172.21.15.47:0/71125086 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c7bc20 0x558be4c89ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.161+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 >> 172.21.15.47:0/71125086 conn(0x558be4ba4e60 msgr2=0x558be4c75080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:47.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.162+0000 7faa277fe700 0 client.25397 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:47.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.166+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 shutdown_connections 2024-05-11T12:33:47.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.166+0000 7faa5914cb80 1 -- 172.21.15.47:0/71125086 wait complete. 2024-05-11T12:33:47.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.168+0000 7faa5914cb80 20 client.25397 trim_cache size 0 max 16384 2024-05-11T12:33:47.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.241+0000 7faa5914cb80 1 Processor -- start 2024-05-11T12:33:47.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.241+0000 7faa5914cb80 1 -- start start 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.241+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.242+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4ca2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.242+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4ca2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.242+0000 7faa5914cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa4013c2a0 con 0x558be4c8a3e0 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.242+0000 7faa477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.243+0000 7faa477fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c780a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:37420/0 (socket says 172.21.15.47:37420) 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.243+0000 7faa477fe700 1 -- 172.21.15.47:0/3651388477 learned_addr learned my addr 172.21.15.47:0/3651388477 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:47.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.243+0000 7faa5914cb80 1 -- 172.21.15.47:0/3651388477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa4013eb20 con 0x558be4c7bc20 2024-05-11T12:33:47.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.243+0000 7faa5914cb80 1 -- 172.21.15.47:0/3651388477 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa4013c910 con 0x558be4c78c00 2024-05-11T12:33:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.245+0000 7faa47fff700 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4ca2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.245+0000 7faa477fe700 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4ca2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.246+0000 7faa47fff700 1 -- 172.21.15.47:0/3651388477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 msgr2=0x558be4c780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.246+0000 7faa47fff700 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.246+0000 7faa47fff700 1 -- 172.21.15.47:0/3651388477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 msgr2=0x558be4ca2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.246+0000 7faa47fff700 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4ca2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.246+0000 7faa47fff700 1 -- 172.21.15.47:0/3651388477 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be4c785e0 con 0x558be4c7bc20 2024-05-11T12:33:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.247+0000 7faa47fff700 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4ca2a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x558be4c739d0 tx=0x7faa3c00cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4adf416963033865 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:47.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.248+0000 7faa31ffb700 1 -- 172.21.15.47:0/3651388477 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c015040 con 0x558be4c7bc20 2024-05-11T12:33:47.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.248+0000 7faa31ffb700 1 -- 172.21.15.47:0/3651388477 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c00ce80 con 0x558be4c7bc20 2024-05-11T12:33:47.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.248+0000 7faa31ffb700 1 -- 172.21.15.47:0/3651388477 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c004550 con 0x558be4c7bc20 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.250+0000 7faa5914cb80 1 -- 172.21.15.47:0/3651388477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 msgr2=0x558be4ca2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.250+0000 7faa5914cb80 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4ca2a60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x558be4c739d0 tx=0x7faa3c00cb10 comp rx=0 tx=0).stop 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.250+0000 7faa5914cb80 1 -- 172.21.15.47:0/3651388477 shutdown_connections 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.251+0000 7faa5914cb80 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4ca2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.251+0000 7faa5914cb80 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c78c00 0x558be4c780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.251+0000 7faa5914cb80 1 --2- 172.21.15.47:0/3651388477 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c7bc20 0x558be4ca2a60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.251+0000 7faa5914cb80 1 -- 172.21.15.47:0/3651388477 >> 172.21.15.47:0/3651388477 conn(0x558be4ba4e60 msgr2=0x558be4c88790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.251+0000 7faa5914cb80 1 -- 172.21.15.47:0/3651388477 shutdown_connections 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.251+0000 7faa5914cb80 1 -- 172.21.15.47:0/3651388477 wait complete. 2024-05-11T12:33:47.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.253+0000 7faa5914cb80 1 -- start start 2024-05-11T12:33:47.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.254+0000 7faa5914cb80 1 Processor -- start 2024-05-11T12:33:47.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.257+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.259+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c7bc20 0x558be4c8e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.259+0000 7faa5914cb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4c8ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.260+0000 7faa5914cb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7faa4013cfe0 con 0x558be4c8a3e0 2024-05-11T12:33:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.260+0000 7faa5914cb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7faa4013e450 con 0x558be4c78c00 2024-05-11T12:33:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.260+0000 7faa5914cb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7faa4013d6b0 con 0x558be4c7bc20 2024-05-11T12:33:47.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.260+0000 7faa5914cb80 10 client.0 _ll_register_callbacks cb 0x558be4b62920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:47.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.262+0000 7faa47fff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c7bc20 0x558be4c8e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.262+0000 7faa477fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.263+0000 7faa477fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c932e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:39374/0 (socket says 172.21.15.47:39374) 2024-05-11T12:33:47.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.263+0000 7faa477fe700 1 -- 172.21.15.47:0/239672122 learned_addr learned my addr 172.21.15.47:0/239672122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:47.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.265+0000 7faa477fe700 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4c8ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.265+0000 7faa477fe700 1 -- 172.21.15.47:0/239672122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c7bc20 msgr2=0x558be4c8e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.265+0000 7faa477fe700 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c7bc20 0x558be4c8e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.265+0000 7faa477fe700 1 -- 172.21.15.47:0/239672122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 msgr2=0x558be4c8ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.265+0000 7faa477fe700 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4c8ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.265+0000 7faa477fe700 1 -- 172.21.15.47:0/239672122 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be4c8f470 con 0x558be4c78c00 2024-05-11T12:33:47.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.267+0000 7faa477fe700 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c932e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faa3400fb00 tx=0x7faa34019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6178e1600846f1fa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:47.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.267+0000 7faa277fe700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:33:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.267+0000 7faa277fe700 1 -- 172.21.15.47:0/239672122 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34003b70 con 0x558be4c78c00 2024-05-11T12:33:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.268+0000 7faa277fe700 1 -- 172.21.15.47:0/239672122 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7faa34021040 con 0x558be4c78c00 2024-05-11T12:33:47.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.268+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558be4c6d260 con 0x558be4c78c00 2024-05-11T12:33:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.270+0000 7faa277fe700 1 -- 172.21.15.47:0/239672122 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7faa34045050 con 0x558be4c78c00 2024-05-11T12:33:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.270+0000 7faa5914cb80 10 client.25403 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:47.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.270+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558be4d056c0 con 0x558be4c78c00 2024-05-11T12:33:47.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.271+0000 7faa277fe700 1 -- 172.21.15.47:0/239672122 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa34003d10 con 0x558be4c78c00 2024-05-11T12:33:47.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.271+0000 7faa277fe700 1 -- 172.21.15.47:0/239672122 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7faa3402a070 con 0x558be4c78c00 2024-05-11T12:33:47.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.271+0000 7faa5914cb80 10 client.25403 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:47.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.271+0000 7faa5914cb80 10 client.25403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:47.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.271+0000 7faa5914cb80 20 client.25403 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:47.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.271+0000 7faa5914cb80 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4d05da0 0x558be4d08200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:47.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.271+0000 7faa5914cb80 4 client.25403 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:33:47.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.272+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558be4c6d260 con 0x558be4d05da0 2024-05-11T12:33:47.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.273+0000 7faa47fff700 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4d05da0 0x558be4d08200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:47.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.276+0000 7faa47fff700 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4d05da0 0x558be4d08200 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e0f7aed9c6e45 server_cookie=37b2f7b62e7ebab0 in_seq=0 out_seq=0 2024-05-11T12:33:47.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.276+0000 7faa277fe700 10 client.25403 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:47.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.279+0000 7faa277fe700 1 -- 172.21.15.47:0/239672122 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558be4c6d260 con 0x558be4d05da0 2024-05-11T12:33:47.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.279+0000 7faa277fe700 10 client.25403 handle_command_reply: tid=0 2024-05-11T12:33:47.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.280+0000 7faa5914cb80 1 client.25403 shutdown 2024-05-11T12:33:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 msgr2=0x558be4c932e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c932e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faa3400fb00 tx=0x7faa34019040 comp rx=0 tx=0).stop 2024-05-11T12:33:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 shutdown_connections 2024-05-11T12:33:47.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558be4d05da0 0x558be4d08200 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558be4c8a3e0 0x558be4c8ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa277fe700 0 client.25403 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:47.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558be4c78c00 0x558be4c932e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 --2- 172.21.15.47:0/239672122 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558be4c7bc20 0x558be4c8e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 >> 172.21.15.47:0/239672122 conn(0x558be4ba4e60 msgr2=0x558be4c85550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 shutdown_connections 2024-05-11T12:33:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.282+0000 7faa5914cb80 1 -- 172.21.15.47:0/239672122 wait complete. 2024-05-11T12:33:47.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.283+0000 7faa5914cb80 20 client.25403 trim_cache size 0 max 16384 2024-05-11T12:33:47.288 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:33:47.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa380b28e0 msgr2=0x7faa380b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa380b28e0 0x7faa380b4da0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faa48115790 tx=0x7faa4807f040 comp rx=0 tx=0).stop 2024-05-11T12:33:47.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa4009aca0 msgr2=0x7faa4012fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa4009aca0 0x7faa4012fd10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa48052c90 tx=0x7faa48075c90 comp rx=0 tx=0).stop 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 shutdown_connections 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7faa4009aca0 0x7faa4012fd10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7faa40094ee0 0x7faa4012b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7faa380b28e0 0x7faa380b4da0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 --2- 172.21.15.47:0/849976192 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7faa400958a0 0x7faa4012b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.301+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 >> 172.21.15.47:0/849976192 conn(0x7faa4000a7a0 msgr2=0x7faa40093c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.304+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 shutdown_connections 2024-05-11T12:33:47.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:47.304+0000 7faa4d654700 1 -- 172.21.15.47:0/849976192 wait complete. 2024-05-11T12:33:48.032 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:47 smithi047 ceph-mon[30375]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 1.61k op/s 2024-05-11T12:33:48.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:47 smithi064 ceph-mon[34537]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 1.61k op/s 2024-05-11T12:33:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:47 smithi050 ceph-mon[34072]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 1.61k op/s 2024-05-11T12:33:49.784 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:49 smithi047 ceph-mon[30375]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 785 MiB/s wr, 1.56k op/s 2024-05-11T12:33:50.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:49 smithi064 ceph-mon[34537]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 785 MiB/s wr, 1.56k op/s 2024-05-11T12:33:50.201 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:49 smithi050 ceph-mon[34072]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 785 MiB/s wr, 1.56k op/s 2024-05-11T12:33:50.576 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:33:50.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:33:51.540 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:33:52.031 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:51 smithi047 ceph-mon[30375]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 805 MiB/s wr, 1.80k op/s 2024-05-11T12:33:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:51 smithi064 ceph-mon[34537]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 805 MiB/s wr, 1.80k op/s 2024-05-11T12:33:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:51 smithi050 ceph-mon[34072]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 805 MiB/s wr, 1.80k op/s 2024-05-11T12:33:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:53 smithi064 ceph-mon[34537]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.75k op/s 2024-05-11T12:33:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:53 smithi050 ceph-mon[34072]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.75k op/s 2024-05-11T12:33:54.237 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:53 smithi047 ceph-mon[30375]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.75k op/s 2024-05-11T12:33:56.019 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:55 smithi047 ceph-mon[30375]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 718 MiB/s wr, 1.51k op/s 2024-05-11T12:33:56.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:55 smithi064 ceph-mon[34537]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 718 MiB/s wr, 1.51k op/s 2024-05-11T12:33:56.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:55 smithi050 ceph-mon[34072]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 718 MiB/s wr, 1.51k op/s 2024-05-11T12:33:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.702+0000 7f43ae108700 1 -- 172.21.15.47:0/1299777451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806c480 msgr2=0x7f43a81103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.702+0000 7f43ae108700 1 --2- 172.21.15.47:0/1299777451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806c480 0x7f43a81103d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f43a000f400 tx=0x7f43a000f710 comp rx=0 tx=0).stop 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.703+0000 7f43ae108700 1 -- 172.21.15.47:0/1299777451 shutdown_connections 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.703+0000 7f43ae108700 1 --2- 172.21.15.47:0/1299777451 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806c480 0x7f43a81103d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.703+0000 7f43ae108700 1 --2- 172.21.15.47:0/1299777451 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a81095b0 0x7f43a8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.703+0000 7f43ae108700 1 --2- 172.21.15.47:0/1299777451 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a806bac0 0x7f43a806bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.703+0000 7f43ae108700 1 -- 172.21.15.47:0/1299777451 >> 172.21.15.47:0/1299777451 conn(0x7f43a80793c0 msgr2=0x7f43a80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.709+0000 7f43ae108700 1 -- 172.21.15.47:0/1299777451 shutdown_connections 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.709+0000 7f43ae108700 1 -- 172.21.15.47:0/1299777451 wait complete. 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.710+0000 7f43ae108700 1 Processor -- start 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.710+0000 7f43ae108700 1 -- start start 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.711+0000 7f43ae108700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806bac0 0x7f43a8074de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.711+0000 7f43ae108700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a806c480 0x7f43a806de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.711+0000 7f43ae108700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a81095b0 0x7f43a806e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.711+0000 7f43ae108700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f43a8111520 con 0x7f43a806bac0 2024-05-11T12:33:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.711+0000 7f43ae108700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81113a0 con 0x7f43a806c480 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.711+0000 7f43ae108700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81116a0 con 0x7f43a81095b0 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.711+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806bac0 0x7f43a8074de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.713+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806bac0 0x7f43a8074de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41058/0 (socket says 172.21.15.47:41058) 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.713+0000 7f43a77fe700 1 -- 172.21.15.47:0/142205304 learned_addr learned my addr 172.21.15.47:0/142205304 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.714+0000 7f43a7fff700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a81095b0 0x7f43a806e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.715+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a806c480 0x7f43a806de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.715+0000 7f43a77fe700 1 -- 172.21.15.47:0/142205304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a81095b0 msgr2=0x7f43a806e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.715+0000 7f43a77fe700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a81095b0 0x7f43a806e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.715+0000 7f43a77fe700 1 -- 172.21.15.47:0/142205304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a806c480 msgr2=0x7f43a806de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.715+0000 7f43a77fe700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a806c480 0x7f43a806de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.715+0000 7f43a77fe700 1 -- 172.21.15.47:0/142205304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a806ea50 con 0x7f43a806bac0 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.716+0000 7f43a77fe700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806bac0 0x7f43a8074de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f439800e9a0 tx=0x7f439800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.716+0000 7f43a4ff9700 1 -- 172.21.15.47:0/142205304 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439800cbd0 con 0x7f43a806bac0 2024-05-11T12:33:56.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.716+0000 7f43a4ff9700 1 -- 172.21.15.47:0/142205304 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439800cd70 con 0x7f43a806bac0 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.717+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a8072910 con 0x7f43a806bac0 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.717+0000 7f43a4ff9700 1 -- 172.21.15.47:0/142205304 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4398018680 con 0x7f43a806bac0 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.718+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a8072e50 con 0x7f43a806bac0 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.723+0000 7f43b899eb80 1 Processor -- start 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.724+0000 7f43b899eb80 1 -- start start 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0f360 0x5620c3c0f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c09db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c16120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43a4ff9700 1 -- 172.21.15.47:0/142205304 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4398010970 con 0x7f43a806bac0 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43b899eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5620c3b538e0 con 0x5620c3c0a2f0 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43b899eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5620c3af2840 con 0x5620c3c0f360 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43b899eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5620c3b4f8a0 con 0x5620c3c09930 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c16120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c16120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41060/0 (socket says 172.21.15.47:41060) 2024-05-11T12:33:56.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.725+0000 7f43a6ffd700 1 -- 172.21.15.47:0/3711379262 learned_addr learned my addr 172.21.15.47:0/3711379262 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.727+0000 7f43a4ff9700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f43900b2a50 0x7f43900b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.727+0000 7f43a6ffd700 1 -- 172.21.15.47:0/3711379262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 msgr2=0x5620c3c09db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.727+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/3711379262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c09db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.727+0000 7f43a6ffd700 1 -- 172.21.15.47:0/3711379262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0f360 msgr2=0x5620c3c0f740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.727+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/3711379262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0f360 0x5620c3c0f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.727+0000 7f43a6ffd700 1 -- 172.21.15.47:0/3711379262 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c3aa0950 con 0x5620c3c0a2f0 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.728+0000 7f43a4ff9700 1 -- 172.21.15.47:0/142205304 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f43980e26e0 con 0x7f43a806bac0 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.729+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/3711379262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c16120 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f439c00bcd0 tx=0x7f439c00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78e80e6db29ef30 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.729+0000 7f43a7fff700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f43900b2a50 0x7f43900b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.729+0000 7f438dffb700 1 -- 172.21.15.47:0/3711379262 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c015070 con 0x5620c3c0a2f0 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.729+0000 7f438dffb700 1 -- 172.21.15.47:0/3711379262 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439c0044e0 con 0x5620c3c0a2f0 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.729+0000 7f438dffb700 1 -- 172.21.15.47:0/3711379262 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c003df0 con 0x5620c3c0a2f0 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.730+0000 7f43b899eb80 1 -- 172.21.15.47:0/3711379262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 msgr2=0x5620c3c16120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.730+0000 7f43b899eb80 1 --2- 172.21.15.47:0/3711379262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c16120 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f439c00bcd0 tx=0x7f439c00f360 comp rx=0 tx=0).stop 2024-05-11T12:33:56.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.731+0000 7f43b899eb80 1 -- 172.21.15.47:0/3711379262 shutdown_connections 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.731+0000 7f43b899eb80 1 --2- 172.21.15.47:0/3711379262 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c16120 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.731+0000 7f43b899eb80 1 --2- 172.21.15.47:0/3711379262 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0f360 0x5620c3c0f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.731+0000 7f43b899eb80 1 --2- 172.21.15.47:0/3711379262 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c09db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.731+0000 7f43b899eb80 1 -- 172.21.15.47:0/3711379262 >> 172.21.15.47:0/3711379262 conn(0x5620c3b2dee0 msgr2=0x5620c3bfba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.731+0000 7f43b899eb80 1 -- 172.21.15.47:0/3711379262 shutdown_connections 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.731+0000 7f43b899eb80 1 -- 172.21.15.47:0/3711379262 wait complete. 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.732+0000 7f43b899eb80 1 -- start start 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.732+0000 7f43b899eb80 1 Processor -- start 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.733+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.733+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.733+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c0f360 0x5620c3c13780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.733+0000 7f43b899eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5620c3c16720 con 0x5620c3c0a2f0 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.733+0000 7f43b899eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5620c3ab9cb0 con 0x5620c3c09930 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.733+0000 7f43b899eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5620c3c168a0 con 0x5620c3c0f360 2024-05-11T12:33:56.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.733+0000 7f43b899eb80 10 client.0 _ll_register_callbacks cb 0x5620c3aeb9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.734+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.734+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:34102/0 (socket says 172.21.15.47:34102) 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.734+0000 7f43a77fe700 1 -- 172.21.15.47:0/1015760047 learned_addr learned my addr 172.21.15.47:0/1015760047 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.734+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.734+0000 7f43a7fff700 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c0f360 0x5620c3c13780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.737+0000 7f43a77fe700 1 -- 172.21.15.47:0/1015760047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c0f360 msgr2=0x5620c3c13780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.737+0000 7f43a77fe700 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c0f360 0x5620c3c13780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.737+0000 7f43a77fe700 1 -- 172.21.15.47:0/1015760047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 msgr2=0x5620c3c13240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.737+0000 7f43a77fe700 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c13240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.737+0000 7f43a77fe700 1 -- 172.21.15.47:0/1015760047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c3aef710 con 0x5620c3c09930 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.737+0000 7f43a77fe700 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4398009230 tx=0x7f43980b2880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=390505c60d693e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.738+0000 7f4386ffd700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.739+0000 7f4386ffd700 1 -- 172.21.15.47:0/1015760047 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43980a9340 con 0x5620c3c09930 2024-05-11T12:33:56.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.739+0000 7f43a7fff700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f43900b2a50 0x7f43900b4f10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f43a000f210 tx=0x7f43a000da90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.739+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439c0044e0 con 0x5620c3c09930 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.739+0000 7f4386ffd700 1 -- 172.21.15.47:0/1015760047 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4398010640 con 0x5620c3c09930 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.740+0000 7f4386ffd700 1 -- 172.21.15.47:0/1015760047 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f43980b2370 con 0x5620c3c09930 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.740+0000 7f43b899eb80 10 client.25412 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.740+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5620c3c210b0 con 0x5620c3c09930 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.740+0000 7f4386ffd700 1 -- 172.21.15.47:0/1015760047 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439c0044e0 con 0x5620c3c09930 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.741+0000 7f4386ffd700 1 -- 172.21.15.47:0/1015760047 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f43980a5050 con 0x5620c3c09930 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.741+0000 7f43b899eb80 10 client.25412 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.741+0000 7f43b899eb80 10 client.25412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.741+0000 7f43b899eb80 20 client.25412 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.741+0000 7f43b899eb80 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c239a0 0x5620c3c25dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.741+0000 7f43b899eb80 4 client.25412 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.741+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620c3b612c0 con 0x5620c3c239a0 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.745+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c239a0 0x5620c3c25dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.746+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c239a0 0x5620c3c25dc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114b8d79a93c6503 server_cookie=4365eb6822ed8fcb in_seq=0 out_seq=0 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.746+0000 7f4386ffd700 10 client.25412 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.749+0000 7f4386ffd700 1 -- 172.21.15.47:0/1015760047 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5620c3b612c0 con 0x5620c3c239a0 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.749+0000 7f4386ffd700 10 client.25412 handle_command_reply: tid=0 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.749+0000 7f43b899eb80 1 client.25412 shutdown 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.750+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 msgr2=0x5620c3c15ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.750+0000 7f43b899eb80 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4398009230 tx=0x7f43980b2880 comp rx=0 tx=0).stop 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.751+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 shutdown_connections 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.751+0000 7f43b899eb80 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c239a0 0x5620c3c25dc0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.751+0000 7f43b899eb80 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c0f360 0x5620c3c13780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.751+0000 7f43b899eb80 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15ea0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.751+0000 7f43b899eb80 1 --2- 172.21.15.47:0/1015760047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c13240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.751+0000 7f4386ffd700 0 client.25412 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.751+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 >> 172.21.15.47:0/1015760047 conn(0x5620c3b2dee0 msgr2=0x5620c3bf64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.752+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 shutdown_connections 2024-05-11T12:33:56.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.759+0000 7f43b899eb80 1 -- 172.21.15.47:0/1015760047 wait complete. 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.767+0000 7f43b899eb80 20 client.25412 trim_cache size 0 max 16384 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.831+0000 7f43b899eb80 1 Processor -- start 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.831+0000 7f43b899eb80 1 -- start start 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.831+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3b2dee0 0x5620c3c1f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.831+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.831+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c15d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.832+0000 7f43b899eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b2670 con 0x5620c3c0a2f0 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.832+0000 7f43b899eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b4ef0 con 0x5620c3c09930 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.832+0000 7f43b899eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b2ce0 con 0x5620c3b2dee0 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.832+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.832+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:34110/0 (socket says 172.21.15.47:34110) 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.832+0000 7f43a6ffd700 1 -- 172.21.15.47:0/637888197 learned_addr learned my addr 172.21.15.47:0/637888197 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.832+0000 7f43a7fff700 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c15d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.834+0000 7f43a7fff700 1 -- 172.21.15.47:0/637888197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3b2dee0 msgr2=0x5620c3c1f7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:33:56.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.834+0000 7f43a7fff700 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3b2dee0 0x5620c3c1f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.834+0000 7f43a7fff700 1 -- 172.21.15.47:0/637888197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 msgr2=0x5620c3c15850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.834+0000 7f43a7fff700 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.834+0000 7f43a7fff700 1 -- 172.21.15.47:0/637888197 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620c3c1fcf0 con 0x5620c3c0a2f0 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.835+0000 7f43a7fff700 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c15d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43a000dfd0 tx=0x7f43a0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795d03b2adc96f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.845+0000 7f438d7fa700 1 -- 172.21.15.47:0/637888197 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a001b030 con 0x5620c3c0a2f0 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.845+0000 7f438d7fa700 1 -- 172.21.15.47:0/637888197 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a001c040 con 0x5620c3c0a2f0 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.846+0000 7f438d7fa700 1 -- 172.21.15.47:0/637888197 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0008ae0 con 0x5620c3c0a2f0 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.847+0000 7f43b899eb80 1 -- 172.21.15.47:0/637888197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 msgr2=0x5620c3c15d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.847+0000 7f43b899eb80 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c15d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f43a000dfd0 tx=0x7f43a0021000 comp rx=0 tx=0).stop 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.850+0000 7f43b899eb80 1 -- 172.21.15.47:0/637888197 shutdown_connections 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.850+0000 7f43b899eb80 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3c0a2f0 0x5620c3c15d90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.850+0000 7f43b899eb80 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3b2dee0 0x5620c3c1f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.850+0000 7f43b899eb80 1 --2- 172.21.15.47:0/637888197 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c09930 0x5620c3c15850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.850+0000 7f43b899eb80 1 -- 172.21.15.47:0/637888197 >> 172.21.15.47:0/637888197 conn(0x5620c3c0eda0 msgr2=0x5620c3bf64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.851+0000 7f43b899eb80 1 -- 172.21.15.47:0/637888197 shutdown_connections 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.855+0000 7f43b899eb80 1 -- 172.21.15.47:0/637888197 wait complete. 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.855+0000 7f43b899eb80 1 -- start start 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 1 Processor -- start 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3b2dee0 0x5620c3c18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c19040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0a2f0 0x5620c3c19580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b33b0 con 0x5620c3b2dee0 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b4820 con 0x5620c3c0a2f0 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f43a81b3a80 con 0x5620c3c09930 2024-05-11T12:33:56.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.856+0000 7f43b899eb80 10 client.0 _ll_register_callbacks cb 0x5620c3aeb9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "55ae36c1-4dc3-44eb-bede-c6ce6f1493a2", 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.857+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c19040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.857+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c19040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52352/0 (socket says 172.21.15.47:52352) 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.857+0000 7f43a6ffd700 1 -- 172.21.15.47:0/2938983644 learned_addr learned my addr 172.21.15.47:0/2938983644 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:33:56.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.857+0000 7f43a7fff700 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0a2f0 0x5620c3c19580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.858+0000 7f43a6ffd700 1 -- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0a2f0 msgr2=0x5620c3c19580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.858+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0a2f0 0x5620c3c19580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.858+0000 7f43a6ffd700 1 -- 172.21.15.47:0/2938983644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3b2dee0 msgr2=0x5620c3c18b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.858+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3b2dee0 0x5620c3c18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.858+0000 7f43a6ffd700 1 -- 172.21.15.47:0/2938983644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620c3c8e480 con 0x5620c3c09930 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.859+0000 7f43a6ffd700 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c19040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c00 tx=0x7f439c00b450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f84d0bc5b34f8a11 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.863+0000 7f4386ffd700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.863+0000 7f4386ffd700 1 -- 172.21.15.47:0/2938983644 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f439c015040 con 0x5620c3c09930 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.863+0000 7f4386ffd700 1 -- 172.21.15.47:0/2938983644 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f439c003c40 con 0x5620c3c09930 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.863+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a001c040 con 0x5620c3c09930 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.864+0000 7f4386ffd700 1 -- 172.21.15.47:0/2938983644 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f439c003e40 con 0x5620c3c09930 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.864+0000 7f43b899eb80 10 client.25435 fetch_fsmap learned FSMap version 20 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.864+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5620c3c8f7b0 con 0x5620c3c09930 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.867+0000 7f4386ffd700 1 -- 172.21.15.47:0/2938983644 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a001c040 con 0x5620c3c09930 2024-05-11T12:33:56.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.867+0000 7f4386ffd700 1 -- 172.21.15.47:0/2938983644 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f439c0295b0 con 0x5620c3c09930 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.868+0000 7f43b899eb80 10 client.25435 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.868+0000 7f43b899eb80 10 client.25435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.868+0000 7f43b899eb80 20 client.25435 populate_metadata read hostname 'smithi047' 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.868+0000 7f43b899eb80 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c0f360 0x5620c3c91da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.868+0000 7f43b899eb80 4 client.25435 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.868+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5620c3b5b360 con 0x5620c3c0f360 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.871+0000 7f43a77fe700 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c0f360 0x5620c3c91da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.889+0000 7f4386ffd700 10 client.25435 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.889+0000 7f43a77fe700 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c0f360 0x5620c3c91da0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20231dcfc26870be server_cookie=f03ca48179b99a7e in_seq=0 out_seq=0 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.898+0000 7f4386ffd700 1 -- 172.21.15.47:0/2938983644 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5620c3b5b360 con 0x5620c3c0f360 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.898+0000 7f4386ffd700 10 client.25435 handle_command_reply: tid=0 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.899+0000 7f43b899eb80 1 client.25435 shutdown 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.900+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 msgr2=0x5620c3c19040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.900+0000 7f43b899eb80 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c19040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f439c000c00 tx=0x7f439c00b450 comp rx=0 tx=0).stop 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.901+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 shutdown_connections 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.902+0000 7f43b899eb80 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5620c3c0f360 0x5620c3c91da0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.902+0000 7f43b899eb80 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5620c3c0a2f0 0x5620c3c19580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.902+0000 7f43b899eb80 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5620c3b2dee0 0x5620c3c18b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.902+0000 7f43b899eb80 1 --2- 172.21.15.47:0/2938983644 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5620c3c09930 0x5620c3c19040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.902+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 >> 172.21.15.47:0/2938983644 conn(0x5620c3c0eda0 msgr2=0x5620c3bf5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.902+0000 7f4386ffd700 0 client.25435 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.903+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 shutdown_connections 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.904+0000 7f43b899eb80 1 -- 172.21.15.47:0/2938983644 wait complete. 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.904+0000 7f43b899eb80 20 client.25435 trim_cache size 0 max 16384 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f43900b2a50 msgr2=0x7f43900b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f43900b2a50 0x7f43900b4f10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f43a000f210 tx=0x7f43a000da90 comp rx=0 tx=0).stop 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806bac0 msgr2=0x7f43a8074de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806bac0 0x7f43a8074de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f439800e9a0 tx=0x7f439800ed60 comp rx=0 tx=0).stop 2024-05-11T12:33:56.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 shutdown_connections 2024-05-11T12:33:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f43a81095b0 0x7f43a806e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f43a806bac0 0x7f43a8074de0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f43900b2a50 0x7f43900b4f10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 --2- 172.21.15.47:0/142205304 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43a806c480 0x7f43a806de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:33:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.905+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 >> 172.21.15.47:0/142205304 conn(0x7f43a80793c0 msgr2=0x7f43a8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:33:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.910+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 shutdown_connections 2024-05-11T12:33:56.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:33:56.910+0000 7f43ae108700 1 -- 172.21.15.47:0/142205304 wait complete. 2024-05-11T12:33:58.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:57 smithi064 ceph-mon[34537]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 817 MiB/s wr, 1.69k op/s 2024-05-11T12:33:58.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:57 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:33:58.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:57 smithi050 ceph-mon[34072]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 817 MiB/s wr, 1.69k op/s 2024-05-11T12:33:58.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:57 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:33:58.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:57 smithi047 ceph-mon[30375]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 817 MiB/s wr, 1.69k op/s 2024-05-11T12:33:58.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:57 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:33:58.840 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:33:59.009 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:58 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:33:59.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:58 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:59.010 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:58 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:33:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:58 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:33:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:58 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:59.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:58 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:33:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:58 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:33:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:58 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:33:59.200 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:58 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:34:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:59 smithi050 ceph-mon[34072]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 810 MiB/s wr, 1.67k op/s 2024-05-11T12:34:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:59 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:33:59 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:00.282 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:59 smithi047 ceph-mon[30375]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 810 MiB/s wr, 1.67k op/s 2024-05-11T12:34:00.282 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:59 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:00.282 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:00.282 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:33:59 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:59 smithi064 ceph-mon[34537]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 810 MiB/s wr, 1.67k op/s 2024-05-11T12:34:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:59 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:33:59 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:34:02.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:01 smithi047 ceph-mon[30375]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 800 MiB/s wr, 1.53k op/s 2024-05-11T12:34:02.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:02.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:01 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:01 smithi064 ceph-mon[34537]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 800 MiB/s wr, 1.53k op/s 2024-05-11T12:34:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:01 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:02.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:01 smithi050 ceph-mon[34072]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 800 MiB/s wr, 1.53k op/s 2024-05-11T12:34:02.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:02.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:01 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:04.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:03 smithi047 ceph-mon[30375]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 836 MiB/s wr, 1.43k op/s 2024-05-11T12:34:04.281 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:03 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:34:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:03 smithi064 ceph-mon[34537]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 836 MiB/s wr, 1.43k op/s 2024-05-11T12:34:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:03 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:34:04.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:03 smithi050 ceph-mon[34072]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 836 MiB/s wr, 1.43k op/s 2024-05-11T12:34:04.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:03 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:34:06.242 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:05 smithi047 ceph-mon[30375]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 716 MiB/s wr, 1.30k op/s 2024-05-11T12:34:06.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:05 smithi064 ceph-mon[34537]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 716 MiB/s wr, 1.30k op/s 2024-05-11T12:34:06.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:05 smithi050 ceph-mon[34072]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 716 MiB/s wr, 1.30k op/s 2024-05-11T12:34:07.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.083+0000 7f68c9bca700 1 -- 172.21.15.47:0/2433112576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68bc0a1680 msgr2=0x7f68bc0a9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.083+0000 7f68c9bca700 1 --2- 172.21.15.47:0/2433112576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68bc0a1680 0x7f68bc0a9250 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68b00099c0 tx=0x7f68b0009cd0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.084+0000 7f68c9bca700 1 -- 172.21.15.47:0/2433112576 shutdown_connections 2024-05-11T12:34:07.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.084+0000 7f68c9bca700 1 --2- 172.21.15.47:0/2433112576 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68bc0a1680 0x7f68bc0a9250 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.084+0000 7f68c9bca700 1 --2- 172.21.15.47:0/2433112576 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68bc09c5a0 0x7f68bc09c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.084+0000 7f68c9bca700 1 --2- 172.21.15.47:0/2433112576 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 0x7f68bc0a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.084+0000 7f68c9bca700 1 -- 172.21.15.47:0/2433112576 >> 172.21.15.47:0/2433112576 conn(0x7f68bc009a70 msgr2=0x7f68bc009e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.094+0000 7f68c9bca700 1 -- 172.21.15.47:0/2433112576 shutdown_connections 2024-05-11T12:34:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.094+0000 7f68c9bca700 1 -- 172.21.15.47:0/2433112576 wait complete. 2024-05-11T12:34:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.096+0000 7f68c9bca700 1 Processor -- start 2024-05-11T12:34:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.096+0000 7f68c9bca700 1 -- start start 2024-05-11T12:34:07.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.097+0000 7f68c9bca700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68bc09c5a0 0x7f68bc138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.097+0000 7f68c9bca700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 0x7f68bc138990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.097+0000 7f68c9bca700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68bc0a1680 0x7f68bc132610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.097+0000 7f68c9bca700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0ac610 con 0x7f68bc09cf50 2024-05-11T12:34:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.097+0000 7f68c9bca700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0ac490 con 0x7f68bc0a1680 2024-05-11T12:34:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.098+0000 7f68c9bca700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f68bc0ac790 con 0x7f68bc09c5a0 2024-05-11T12:34:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.098+0000 7f68c3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 0x7f68bc138990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.098+0000 7f68c93c9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68bc0a1680 0x7f68bc132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.098+0000 7f68c3fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 0x7f68bc138990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47532/0 (socket says 172.21.15.47:47532) 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.098+0000 7f68c3fff700 1 -- 172.21.15.47:0/1269565366 learned_addr learned my addr 172.21.15.47:0/1269565366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.099+0000 7f68c3fff700 1 -- 172.21.15.47:0/1269565366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68bc09c5a0 msgr2=0x7f68bc138450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.099+0000 7f68c3fff700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68bc09c5a0 0x7f68bc138450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.099+0000 7f68c3fff700 1 -- 172.21.15.47:0/1269565366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68bc0a1680 msgr2=0x7f68bc132610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.099+0000 7f68c3fff700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68bc0a1680 0x7f68bc132610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.099+0000 7f68c3fff700 1 -- 172.21.15.47:0/1269565366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68bc132d70 con 0x7f68bc09cf50 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.100+0000 7f68c3fff700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 0x7f68bc138990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f68c4068d30 tx=0x7f68c406c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.101+0000 7f68c1ffb700 1 -- 172.21.15.47:0/1269565366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4075e50 con 0x7f68bc09cf50 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.101+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68bc133060 con 0x7f68bc09cf50 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.101+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68bc13f320 con 0x7f68bc09cf50 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.102+0000 7f68c1ffb700 1 -- 172.21.15.47:0/1269565366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c4084070 con 0x7f68bc09cf50 2024-05-11T12:34:07.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.102+0000 7f68c1ffb700 1 -- 172.21.15.47:0/1269565366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c407e7b0 con 0x7f68bc09cf50 2024-05-11T12:34:07.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.108+0000 7f68c1ffb700 1 -- 172.21.15.47:0/1269565366 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f68c407c050 con 0x7f68bc09cf50 2024-05-11T12:34:07.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.109+0000 7f68d56c2b80 1 Processor -- start 2024-05-11T12:34:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.111+0000 7f68d56c2b80 1 -- start start 2024-05-11T12:34:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.111+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf2803e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.112+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf280920 0x557bcf27a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.112+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf27ab30 0x557bcf27af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.112+0000 7f68d56c2b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557bcf1cc940 con 0x557bcf280000 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.112+0000 7f68d56c2b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557bcf131da0 con 0x557bcf27ab30 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.112+0000 7f68d56c2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557bcf1c8900 con 0x557bcf280920 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.113+0000 7f68c3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf27ab30 0x557bcf27af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.113+0000 7f68c3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf27ab30 0x557bcf27af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:34730/0 (socket says 172.21.15.47:34730) 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.113+0000 7f68c3fff700 1 -- 172.21.15.47:0/1489915809 learned_addr learned my addr 172.21.15.47:0/1489915809 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.113+0000 7f68c1ffb700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f68b40b2bb0 0x7f68b40b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.114+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf280920 0x557bcf27a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.114+0000 7f68c1ffb700 1 -- 172.21.15.47:0/1269565366 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f68c406a5b0 con 0x7f68bc09cf50 2024-05-11T12:34:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.115+0000 7f68c3fff700 1 -- 172.21.15.47:0/1489915809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf280920 msgr2=0x557bcf27a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.115+0000 7f68c3fff700 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf280920 0x557bcf27a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.115+0000 7f68c3fff700 1 -- 172.21.15.47:0/1489915809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 msgr2=0x557bcf2803e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.115+0000 7f68c3fff700 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf2803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.115+0000 7f68c3fff700 1 -- 172.21.15.47:0/1489915809 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bcf1198d0 con 0x557bcf27ab30 2024-05-11T12:34:07.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.117+0000 7f68c3fff700 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf27ab30 0x557bcf27af90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68c4068f50 tx=0x7f68c40695f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e7e7a4fc928151f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.117+0000 7f68aeffd700 1 -- 172.21.15.47:0/1489915809 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4118670 con 0x557bcf27ab30 2024-05-11T12:34:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.118+0000 7f68aeffd700 1 -- 172.21.15.47:0/1489915809 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c4117f70 con 0x557bcf27ab30 2024-05-11T12:34:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.118+0000 7f68aeffd700 1 -- 172.21.15.47:0/1489915809 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c41181d0 con 0x557bcf27ab30 2024-05-11T12:34:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.118+0000 7f68c93c9700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f68b40b2bb0 0x7f68b40b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.119+0000 7f68c93c9700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f68b40b2bb0 0x7f68b40b5070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009fd0 tx=0x7f68b0005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:07.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.120+0000 7f68d56c2b80 1 -- 172.21.15.47:0/1489915809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf27ab30 msgr2=0x557bcf27af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.120+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf27ab30 0x557bcf27af90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68c4068f50 tx=0x7f68c40695f0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.124+0000 7f68d56c2b80 1 -- 172.21.15.47:0/1489915809 shutdown_connections 2024-05-11T12:34:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.124+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf27ab30 0x557bcf27af90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.124+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf2803e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.124+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/1489915809 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf280920 0x557bcf27a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.124+0000 7f68d56c2b80 1 -- 172.21.15.47:0/1489915809 >> 172.21.15.47:0/1489915809 conn(0x557bcf1a6b70 msgr2=0x557bcf275090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.125+0000 7f68d56c2b80 1 -- 172.21.15.47:0/1489915809 shutdown_connections 2024-05-11T12:34:07.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.125+0000 7f68d56c2b80 1 -- 172.21.15.47:0/1489915809 wait complete. 2024-05-11T12:34:07.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.126+0000 7f68d56c2b80 1 -- start start 2024-05-11T12:34:07.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.127+0000 7f68d56c2b80 1 Processor -- start 2024-05-11T12:34:07.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.128+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf2866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.128+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf283a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf283f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68c8bc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf2866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68c8bc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf2866a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49430/0 (socket says 172.21.15.47:49430) 2024-05-11T12:34:07.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68c8bc8700 1 -- 172.21.15.47:0/3417204933 learned_addr learned my addr 172.21.15.47:0/3417204933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:07.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x557bcf287380 con 0x557bcf280000 2024-05-11T12:34:07.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x557bcf185ed0 con 0x557bcf280920 2024-05-11T12:34:07.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x557bcf287500 con 0x557bcf27ab30 2024-05-11T12:34:07.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.129+0000 7f68d56c2b80 10 client.0 _ll_register_callbacks cb 0x557bcf164950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.132+0000 7f68c3fff700 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf283a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.133+0000 7f68c3fff700 1 -- 172.21.15.47:0/3417204933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 msgr2=0x557bcf2866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.133+0000 7f68c3fff700 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf2866a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.133+0000 7f68c3fff700 1 -- 172.21.15.47:0/3417204933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 msgr2=0x557bcf283f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.133+0000 7f68c3fff700 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf283f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.133+0000 7f68c3fff700 1 -- 172.21.15.47:0/3417204933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bcf183760 con 0x557bcf280000 2024-05-11T12:34:07.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.135+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf2866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:34:07.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.142+0000 7f68c3fff700 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf283a40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x557bcf287090 tx=0x7f68c40696d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb53f095fa6410e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:07.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.142+0000 7f68a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:34:07.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.147+0000 7f68a3fff700 1 -- 172.21.15.47:0/3417204933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c411c320 con 0x557bcf280000 2024-05-11T12:34:07.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.147+0000 7f68a3fff700 1 -- 172.21.15.47:0/3417204933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f68c4120940 con 0x557bcf280000 2024-05-11T12:34:07.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.147+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c4117f70 con 0x557bcf280000 2024-05-11T12:34:07.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.151+0000 7f68a3fff700 1 -- 172.21.15.47:0/3417204933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f68c4120dd0 con 0x557bcf280000 2024-05-11T12:34:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.152+0000 7f68d56c2b80 10 client.15855 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:07.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.152+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557bcf291c90 con 0x557bcf280000 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68a3fff700 1 -- 172.21.15.47:0/3417204933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c4118670 con 0x557bcf280000 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68a3fff700 1 -- 172.21.15.47:0/3417204933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f68c411b070 con 0x557bcf280000 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68d56c2b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68d56c2b80 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68d56c2b80 20 client.15855 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf294160 0x557bcf296550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68d56c2b80 4 client.15855 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:34:07.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.159+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f68c4117f70 con 0x557bcf294160 2024-05-11T12:34:07.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.164+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf294160 0x557bcf296550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.170+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf294160 0x557bcf296550 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ca5b83075321cd server_cookie=723c9495058cef7e in_seq=0 out_seq=0 2024-05-11T12:34:07.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.171+0000 7f68a3fff700 10 client.15855 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.175+0000 7f68a3fff700 1 -- 172.21.15.47:0/3417204933 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f68c4117f70 con 0x557bcf294160 2024-05-11T12:34:07.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.175+0000 7f68a3fff700 10 client.15855 handle_command_reply: tid=0 2024-05-11T12:34:07.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.176+0000 7f68d56c2b80 1 client.15855 shutdown 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 msgr2=0x557bcf283a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf283a40 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x557bcf287090 tx=0x7f68c40696d0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 shutdown_connections 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf294160 0x557bcf296550 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68a3fff700 0 client.15855 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf283f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf2866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3417204933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf280000 0x557bcf283a40 secure :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0x557bcf287090 tx=0x7f68c40696d0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.177+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 >> 172.21.15.47:0/3417204933 conn(0x557bcf1a6b70 msgr2=0x557bcf274660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:07.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.179+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 shutdown_connections 2024-05-11T12:34:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.183+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3417204933 wait complete. 2024-05-11T12:34:07.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.183+0000 7f68d56c2b80 20 client.15855 trim_cache size 0 max 16384 2024-05-11T12:34:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.231+0000 7f68d56c2b80 1 Processor -- start 2024-05-11T12:34:07.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.235+0000 7f68d56c2b80 1 -- start start 2024-05-11T12:34:07.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.235+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf27ab30 0x557bcf27a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.236+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf27e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.236+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf299570 0x557bcf2999f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.236+0000 7f68d56c2b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f68a800b710 con 0x557bcf27ab30 2024-05-11T12:34:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.236+0000 7f68d56c2b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68a800df90 con 0x557bcf280920 2024-05-11T12:34:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.236+0000 7f68d56c2b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f68a800bd80 con 0x557bcf299570 2024-05-11T12:34:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.236+0000 7f68c93c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf299570 0x557bcf2999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.237+0000 7f68c93c9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf299570 0x557bcf2999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49440/0 (socket says 172.21.15.47:49440) 2024-05-11T12:34:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.237+0000 7f68c93c9700 1 -- 172.21.15.47:0/4167426416 learned_addr learned my addr 172.21.15.47:0/4167426416 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.237+0000 7f68c3fff700 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf27e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.237+0000 7f68c93c9700 1 -- 172.21.15.47:0/4167426416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 msgr2=0x557bcf27e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.237+0000 7f68c93c9700 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf27e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.237+0000 7f68c93c9700 1 -- 172.21.15.47:0/4167426416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf27ab30 msgr2=0x557bcf27a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.236+0000 7f68c8bc8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf27ab30 0x557bcf27a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.238+0000 7f68c93c9700 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf27ab30 0x557bcf27a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.238+0000 7f68c93c9700 1 -- 172.21.15.47:0/4167426416 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bcf2848f0 con 0x557bcf299570 2024-05-11T12:34:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.238+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf27ab30 0x557bcf27a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:34:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.239+0000 7f68c93c9700 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf299570 0x557bcf2999f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f68b0015000 tx=0x7f68b0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb0c2eebcdb25d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.240+0000 7f68ae7fc700 1 -- 172.21.15.47:0/4167426416 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b000b790 con 0x557bcf299570 2024-05-11T12:34:07.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.240+0000 7f68ae7fc700 1 -- 172.21.15.47:0/4167426416 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b002a030 con 0x557bcf299570 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.241+0000 7f68ae7fc700 1 -- 172.21.15.47:0/4167426416 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0024b50 con 0x557bcf299570 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 -- 172.21.15.47:0/4167426416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf299570 msgr2=0x557bcf2999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf299570 0x557bcf2999f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f68b0015000 tx=0x7f68b0005fd0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 -- 172.21.15.47:0/4167426416 shutdown_connections 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf299570 0x557bcf2999f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf27ab30 0x557bcf27a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/4167426416 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf27e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 -- 172.21.15.47:0/4167426416 >> 172.21.15.47:0/4167426416 conn(0x557bcf1a6b70 msgr2=0x557bcf27da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.242+0000 7f68d56c2b80 1 -- 172.21.15.47:0/4167426416 shutdown_connections 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.243+0000 7f68d56c2b80 1 -- 172.21.15.47:0/4167426416 wait complete. 2024-05-11T12:34:07.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.243+0000 7f68d56c2b80 1 -- start start 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.243+0000 7f68d56c2b80 1 Processor -- start 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.244+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf298c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.244+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf297010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.244+0000 7f68d56c2b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf299570 0x557bcf297550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.244+0000 7f68c8bc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf298c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.244+0000 7f68c8bc8700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf298c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:49452/0 (socket says 172.21.15.47:49452) 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68c8bc8700 1 -- 172.21.15.47:0/3521333191 learned_addr learned my addr 172.21.15.47:0/3521333191 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f68a800c450 con 0x557bcf299570 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f68a800d8c0 con 0x557bcf280920 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f68a800cb20 con 0x557bcf27ab30 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68d56c2b80 10 client.0 _ll_register_callbacks cb 0x557bcf164950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68c8bc8700 1 -- 172.21.15.47:0/3521333191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 msgr2=0x557bcf297010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf297010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68c8bc8700 1 -- 172.21.15.47:0/3521333191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf299570 msgr2=0x557bcf297550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf299570 0x557bcf297550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.245+0000 7f68c8bc8700 1 -- 172.21.15.47:0/3521333191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bcf297b40 con 0x557bcf27ab30 2024-05-11T12:34:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68c8bc8700 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf298c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x557bcf287510 tx=0x7f68b8009d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3155130a4b569670 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:07.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:34:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68a3fff700 1 -- 172.21.15.47:0/3521333191 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b800f040 con 0x557bcf27ab30 2024-05-11T12:34:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b002a030 con 0x557bcf27ab30 2024-05-11T12:34:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68a3fff700 1 -- 172.21.15.47:0/3521333191 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f68b8016bc0 con 0x557bcf27ab30 2024-05-11T12:34:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68a3fff700 1 -- 172.21.15.47:0/3521333191 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f68b8016de0 con 0x557bcf27ab30 2024-05-11T12:34:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68d56c2b80 10 client.25447 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.246+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557bcf2f5450 con 0x557bcf27ab30 2024-05-11T12:34:07.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68a3fff700 1 -- 172.21.15.47:0/3521333191 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68b002a030 con 0x557bcf27ab30 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68a3fff700 1 -- 172.21.15.47:0/3521333191 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f68b8029410 con 0x557bcf27ab30 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68d56c2b80 10 client.25447 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68d56c2b80 10 client.25447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68d56c2b80 20 client.25447 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf2f5b60 0x557bcf2f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68d56c2b80 4 client.25447 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.247+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557bcf1da320 con 0x557bcf2f5b60 2024-05-11T12:34:07.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.249+0000 7f68c3fff700 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf2f5b60 0x557bcf2f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:07.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.254+0000 7f68c3fff700 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf2f5b60 0x557bcf2f7fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c8920e1cb806f9 server_cookie=27cd9abc379d9a78 in_seq=0 out_seq=0 2024-05-11T12:34:07.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.254+0000 7f68a3fff700 10 client.25447 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:07.263 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.259+0000 7f68a3fff700 1 -- 172.21.15.47:0/3521333191 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557bcf1da320 con 0x557bcf2f5b60 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.259+0000 7f68a3fff700 10 client.25447 handle_command_reply: tid=0 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.259+0000 7f68d56c2b80 1 client.25447 shutdown 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 msgr2=0x557bcf298c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf298c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x557bcf287510 tx=0x7f68b8009d10 comp rx=0 tx=0).stop 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 shutdown_connections 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x557bcf2f5b60 0x557bcf2f7fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x557bcf299570 0x557bcf297550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x557bcf27ab30 0x557bcf298c60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 --2- 172.21.15.47:0/3521333191 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x557bcf280920 0x557bcf297010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.260+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 >> 172.21.15.47:0/3521333191 conn(0x557bcf1a6b70 msgr2=0x557bcf2822f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:07.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.261+0000 7f68a3fff700 0 client.25447 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:07.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.261+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 shutdown_connections 2024-05-11T12:34:07.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.261+0000 7f68d56c2b80 1 -- 172.21.15.47:0/3521333191 wait complete. 2024-05-11T12:34:07.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.261+0000 7f68d56c2b80 20 client.25447 trim_cache size 0 max 16384 2024-05-11T12:34:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.265+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f68b40b2bb0 msgr2=0x7f68b40b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.265+0000 7f68c9bca700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f68b40b2bb0 0x7f68b40b5070 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009fd0 tx=0x7f68b0005da0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.265+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 msgr2=0x7f68bc138990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.265+0000 7f68c9bca700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 0x7f68bc138990 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f68c4068d30 tx=0x7f68c406c850 comp rx=0 tx=0).stop 2024-05-11T12:34:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.267+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 shutdown_connections 2024-05-11T12:34:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.267+0000 7f68c9bca700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f68b40b2bb0 0x7f68b40b5070 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.267+0000 7f68c9bca700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68bc0a1680 0x7f68bc132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.267+0000 7f68c9bca700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f68bc09c5a0 0x7f68bc138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.267+0000 7f68c9bca700 1 --2- 172.21.15.47:0/1269565366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68bc09cf50 0x7f68bc138990 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:07.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.267+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 >> 172.21.15.47:0/1269565366 conn(0x7f68bc009a70 msgr2=0x7f68bc09a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:07.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.269+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 shutdown_connections 2024-05-11T12:34:07.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:07.269+0000 7f68c9bca700 1 -- 172.21.15.47:0/1269565366 wait complete. 2024-05-11T12:34:08.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:07 smithi047 ceph-mon[30375]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-05-11T12:34:08.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:07 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:08.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:07 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:07 smithi064 ceph-mon[34537]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-05-11T12:34:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:07 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:07 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:07 smithi050 ceph-mon[34072]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 MiB/s wr, 1.58k op/s 2024-05-11T12:34:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:07 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:07 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:09.157 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:34:09.157 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:34:10.306 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:34:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:10 smithi064 ceph-mon[34537]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.51k op/s 2024-05-11T12:34:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:34:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:10 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:34:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:10 smithi050 ceph-mon[34072]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.51k op/s 2024-05-11T12:34:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:34:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:10 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:34:10.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:10 smithi047 ceph-mon[30375]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 803 MiB/s wr, 1.51k op/s 2024-05-11T12:34:10.529 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:34:10.530 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:10 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:34:11.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:11.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:11.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:34:12.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:12 smithi047 ceph-mon[30375]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 1.44k op/s 2024-05-11T12:34:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:12 smithi064 ceph-mon[34537]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 1.44k op/s 2024-05-11T12:34:12.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:12 smithi050 ceph-mon[34072]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 794 MiB/s wr, 1.44k op/s 2024-05-11T12:34:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:14 smithi064 ceph-mon[34537]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 863 MiB/s wr, 1.57k op/s 2024-05-11T12:34:14.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:14 smithi050 ceph-mon[34072]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 863 MiB/s wr, 1.57k op/s 2024-05-11T12:34:14.468 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:14 smithi047 ceph-mon[30375]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 863 MiB/s wr, 1.57k op/s 2024-05-11T12:34:15.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.322+0000 7fa0c860c700 1 -- 172.21.15.47:0/2245852801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 msgr2=0x7fa0c0075b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:15.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.322+0000 7fa0c860c700 1 --2- 172.21.15.47:0/2245852801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0075b60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa0b801c0d0 tx=0x7fa0b801c3e0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.323+0000 7fa0c860c700 1 -- 172.21.15.47:0/2245852801 shutdown_connections 2024-05-11T12:34:15.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.323+0000 7fa0c860c700 1 --2- 172.21.15.47:0/2245852801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0c0114720 0x7fa0c0116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.323+0000 7fa0c860c700 1 --2- 172.21.15.47:0/2245852801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0075b60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.323+0000 7fa0c860c700 1 --2- 172.21.15.47:0/2245852801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0c00760a0 0x7fa0c01141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.323+0000 7fa0c860c700 1 -- 172.21.15.47:0/2245852801 >> 172.21.15.47:0/2245852801 conn(0x7fa0c007a6b0 msgr2=0x7fa0c007aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.327+0000 7fa0c860c700 1 -- 172.21.15.47:0/2245852801 shutdown_connections 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.327+0000 7fa0c860c700 1 -- 172.21.15.47:0/2245852801 wait complete. 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.328+0000 7fa0c860c700 1 Processor -- start 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.330+0000 7fa0c860c700 1 -- start start 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.330+0000 7fa0c860c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0072300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.330+0000 7fa0c860c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0c00760a0 0x7fa0c0072840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.330+0000 7fa0c860c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0c0114720 0x7fa0c0072d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.330+0000 7fa0c860c700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010a1d0 con 0x7fa0c0075780 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.330+0000 7fa0c860c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010a050 con 0x7fa0c00760a0 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.330+0000 7fa0c860c700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010a350 con 0x7fa0c0114720 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.331+0000 7fa0c63a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:15.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.331+0000 7fa0c63a8700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0072300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47590/0 (socket says 172.21.15.47:47590) 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.331+0000 7fa0c63a8700 1 -- 172.21.15.47:0/1602387066 learned_addr learned my addr 172.21.15.47:0/1602387066 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.332+0000 7fa0c63a8700 1 -- 172.21.15.47:0/1602387066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0c0114720 msgr2=0x7fa0c0072d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.332+0000 7fa0c63a8700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0c0114720 0x7fa0c0072d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.332+0000 7fa0c63a8700 1 -- 172.21.15.47:0/1602387066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0c00760a0 msgr2=0x7fa0c0072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.332+0000 7fa0c63a8700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0c00760a0 0x7fa0c0072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.332+0000 7fa0c63a8700 1 -- 172.21.15.47:0/1602387066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c0113990 con 0x7fa0c0075780 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.333+0000 7fa0c63a8700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0072300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8004fb0 tx=0x7fa0b801cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.334+0000 7fa0b77fe700 1 -- 172.21.15.47:0/1602387066 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8031030 con 0x7fa0c0075780 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.334+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c0113c20 con 0x7fa0c0075780 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.334+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c0114100 con 0x7fa0c0075780 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.334+0000 7fa0b77fe700 1 -- 172.21.15.47:0/1602387066 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b8023c40 con 0x7fa0c0075780 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.334+0000 7fa0b77fe700 1 -- 172.21.15.47:0/1602387066 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b802daa0 con 0x7fa0c0075780 2024-05-11T12:34:15.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.343+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a4005330 con 0x7fa0c0075780 2024-05-11T12:34:15.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.344+0000 7fa0b77fe700 1 -- 172.21.15.47:0/1602387066 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa0b8034030 con 0x7fa0c0075780 2024-05-11T12:34:15.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.345+0000 7fa0b77fe700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0ac0b2a80 0x7fa0ac0b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:15.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.346+0000 7fa0b77fe700 1 -- 172.21.15.47:0/1602387066 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fa0b80f4150 con 0x7fa0c0075780 2024-05-11T12:34:15.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.346+0000 7fa0c5ba7700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0ac0b2a80 0x7fa0ac0b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:15.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.347+0000 7fa0c5ba7700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0ac0b2a80 0x7fa0ac0b4f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0073ac0 tx=0x7fa0b0005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:15.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.359+0000 7fa0b77fe700 1 -- 172.21.15.47:0/1602387066 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0b80bbdd0 con 0x7fa0c0075780 2024-05-11T12:34:15.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.968+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0a4005620 con 0x7fa0c0075780 2024-05-11T12:34:15.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.971+0000 7fa0b77fe700 1 -- 172.21.15.47:0/1602387066 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fa0b80bb6d0 con 0x7fa0c0075780 2024-05-11T12:34:15.978 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:34:15.978 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:34:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.981+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0ac0b2a80 msgr2=0x7fa0ac0b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.981+0000 7fa0c860c700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0ac0b2a80 0x7fa0ac0b4f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0073ac0 tx=0x7fa0b0005c80 comp rx=0 tx=0).stop 2024-05-11T12:34:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.981+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 msgr2=0x7fa0c0072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:15.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.981+0000 7fa0c860c700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0072300 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8004fb0 tx=0x7fa0b801cf20 comp rx=0 tx=0).stop 2024-05-11T12:34:15.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 shutdown_connections 2024-05-11T12:34:15.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fa0c0114720 0x7fa0c0072d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0c0075780 0x7fa0c0072300 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fa0ac0b2a80 0x7fa0ac0b4f40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 --2- 172.21.15.47:0/1602387066 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa0c00760a0 0x7fa0c0072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:15.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 >> 172.21.15.47:0/1602387066 conn(0x7fa0c007a6b0 msgr2=0x7fa0c0105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:15.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 shutdown_connections 2024-05-11T12:34:15.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:15.984+0000 7fa0c860c700 1 -- 172.21.15.47:0/1602387066 wait complete. 2024-05-11T12:34:15.989 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:34:16.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:16 smithi047 ceph-mon[30375]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 720 MiB/s wr, 1.39k op/s 2024-05-11T12:34:16.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:16 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1602387066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:34:16.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:16 smithi064 ceph-mon[34537]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 720 MiB/s wr, 1.39k op/s 2024-05-11T12:34:16.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:16 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1602387066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:34:16.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:16 smithi050 ceph-mon[34072]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 720 MiB/s wr, 1.39k op/s 2024-05-11T12:34:16.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:16 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1602387066' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:34:18.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:18 smithi064 ceph-mon[34537]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.90k op/s 2024-05-11T12:34:18.393 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:18 smithi047 ceph-mon[30375]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.90k op/s 2024-05-11T12:34:18.398 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:34:18.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:18 smithi050 ceph-mon[34072]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 844 MiB/s wr, 1.90k op/s 2024-05-11T12:34:19.369 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:34:19.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:19 smithi050 ceph-mon[34072]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 792 MiB/s wr, 1.96k op/s 2024-05-11T12:34:19.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:19 smithi047 ceph-mon[30375]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 792 MiB/s wr, 1.96k op/s 2024-05-11T12:34:19.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:19 smithi064 ceph-mon[34537]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 792 MiB/s wr, 1.96k op/s 2024-05-11T12:34:21.699 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:21 smithi050 ceph-mon[34072]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 772 MiB/s wr, 1.93k op/s 2024-05-11T12:34:21.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:21 smithi047 ceph-mon[30375]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 772 MiB/s wr, 1.93k op/s 2024-05-11T12:34:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:21 smithi064 ceph-mon[34537]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 772 MiB/s wr, 1.93k op/s 2024-05-11T12:34:23.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:23 smithi047 ceph-mon[30375]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 832 MiB/s wr, 2.14k op/s 2024-05-11T12:34:23.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:23 smithi064 ceph-mon[34537]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 832 MiB/s wr, 2.14k op/s 2024-05-11T12:34:23.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:23 smithi050 ceph-mon[34072]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 832 MiB/s wr, 2.14k op/s 2024-05-11T12:34:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.091+0000 7fdbd3af0700 1 -- 172.21.15.47:0/4206557195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbcc103fa0 msgr2=0x7fdbcc110480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.091+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/4206557195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc110480 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009990 tx=0x7fdbc8009ca0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.091+0000 7fdbd3af0700 1 -- 172.21.15.47:0/4206557195 shutdown_connections 2024-05-11T12:34:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.091+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/4206557195 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc110480 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.091+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/4206557195 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc108510 0x7fdbcc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.091+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/4206557195 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbcc103600 0x7fdbcc103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.091+0000 7fdbd3af0700 1 -- 172.21.15.47:0/4206557195 >> 172.21.15.47:0/4206557195 conn(0x7fdbcc07a6b0 msgr2=0x7fdbcc07aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.092+0000 7fdbd3af0700 1 -- 172.21.15.47:0/4206557195 shutdown_connections 2024-05-11T12:34:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.092+0000 7fdbd3af0700 1 -- 172.21.15.47:0/4206557195 wait complete. 2024-05-11T12:34:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.093+0000 7fdbd3af0700 1 Processor -- start 2024-05-11T12:34:25.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.094+0000 7fdbd3af0700 1 -- start start 2024-05-11T12:34:25.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.094+0000 7fdbd3af0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbcc103600 0x7fdbcc1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.094+0000 7fdbd3af0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.095+0000 7fdbd3af0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbcc108510 0x7fdbcc10f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.095+0000 7fdbd3af0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc1138a0 con 0x7fdbcc103600 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.095+0000 7fdbd3af0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc113720 con 0x7fdbcc103fa0 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.095+0000 7fdbd3af0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbcc113a20 con 0x7fdbcc108510 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.096+0000 7fdbd108b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.096+0000 7fdbd208d700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbcc108510 0x7fdbcc10f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.096+0000 7fdbd108b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc1a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:36230/0 (socket says 172.21.15.47:36230) 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.096+0000 7fdbd108b700 1 -- 172.21.15.47:0/3419967248 learned_addr learned my addr 172.21.15.47:0/3419967248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.097+0000 7fdbd108b700 1 -- 172.21.15.47:0/3419967248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbcc108510 msgr2=0x7fdbcc10f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.097+0000 7fdbd108b700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbcc108510 0x7fdbcc10f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.097+0000 7fdbd108b700 1 -- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbcc103600 msgr2=0x7fdbcc1a0a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:34:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.097+0000 7fdbd108b700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbcc103600 0x7fdbcc1a0a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.097+0000 7fdbd108b700 1 -- 172.21.15.47:0/3419967248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbcc10ff70 con 0x7fdbcc103fa0 2024-05-11T12:34:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.097+0000 7fdbd108b700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc1a0f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00b530 tx=0x7fdbbc00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.098+0000 7fdbc2ffd700 1 -- 172.21.15.47:0/3419967248 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc015020 con 0x7fdbcc103fa0 2024-05-11T12:34:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.098+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbcc110260 con 0x7fdbcc103fa0 2024-05-11T12:34:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.098+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbcc19c550 con 0x7fdbcc103fa0 2024-05-11T12:34:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.099+0000 7fdbc2ffd700 1 -- 172.21.15.47:0/3419967248 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbbc011ce0 con 0x7fdbcc103fa0 2024-05-11T12:34:25.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.100+0000 7fdbc2ffd700 1 -- 172.21.15.47:0/3419967248 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc010d80 con 0x7fdbcc103fa0 2024-05-11T12:34:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.105+0000 7fdbc2ffd700 1 -- 172.21.15.47:0/3419967248 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fdbbc018020 con 0x7fdbcc103fa0 2024-05-11T12:34:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.106+0000 7fdbc2ffd700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdbb80b2a90 0x7fdbb80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.107+0000 7fdbc2ffd700 1 -- 172.21.15.47:0/3419967248 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fdbbc0e29f0 con 0x7fdbcc103fa0 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.113+0000 7fdbd188c700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdbb80b2a90 0x7fdbb80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.113+0000 7fdbde386b80 1 Processor -- start 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.114+0000 7fdbde386b80 1 -- start start 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.114+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 0x5638dd553eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.114+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd54e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.114+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd55a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.114+0000 7fdbde386b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5638dd4a0860 con 0x5638dd553ad0 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.114+0000 7fdbde386b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638dd43f7c0 con 0x5638dd54ea60 2024-05-11T12:34:25.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.114+0000 7fdbde386b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5638dd49c820 con 0x5638dd54e0a0 2024-05-11T12:34:25.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.115+0000 7fdbd188c700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdbb80b2a90 0x7fdbb80b4f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc1047e0 tx=0x7fdbc4009250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.118+0000 7fdbd208d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd55a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.118+0000 7fdbd208d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd55a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:36242/0 (socket says 172.21.15.47:36242) 2024-05-11T12:34:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.118+0000 7fdbd208d700 1 -- 172.21.15.47:0/4183460825 learned_addr learned my addr 172.21.15.47:0/4183460825 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.122+0000 7fdbd208d700 1 -- 172.21.15.47:0/4183460825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 msgr2=0x5638dd54e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.122+0000 7fdbd208d700 1 --2- 172.21.15.47:0/4183460825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd54e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.122+0000 7fdbd208d700 1 -- 172.21.15.47:0/4183460825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 msgr2=0x5638dd553eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.122+0000 7fdbd208d700 1 --2- 172.21.15.47:0/4183460825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 0x5638dd553eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.122+0000 7fdbd208d700 1 -- 172.21.15.47:0/4183460825 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd3ed8d0 con 0x5638dd54ea60 2024-05-11T12:34:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.122+0000 7fdbd208d700 1 --2- 172.21.15.47:0/4183460825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd55a890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8003700 tx=0x7fdbc8003730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365889b4040fa513 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.132+0000 7fdbb7fff700 1 -- 172.21.15.47:0/4183460825 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8026070 con 0x5638dd54ea60 2024-05-11T12:34:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.133+0000 7fdbb7fff700 1 -- 172.21.15.47:0/4183460825 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbc80050d0 con 0x5638dd54ea60 2024-05-11T12:34:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.133+0000 7fdbb7fff700 1 -- 172.21.15.47:0/4183460825 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8003cc0 con 0x5638dd54ea60 2024-05-11T12:34:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.134+0000 7fdbde386b80 1 -- 172.21.15.47:0/4183460825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 msgr2=0x5638dd55a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.134+0000 7fdbde386b80 1 --2- 172.21.15.47:0/4183460825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd55a890 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8003700 tx=0x7fdbc8003730 comp rx=0 tx=0).stop 2024-05-11T12:34:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.135+0000 7fdbde386b80 1 -- 172.21.15.47:0/4183460825 shutdown_connections 2024-05-11T12:34:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.135+0000 7fdbde386b80 1 --2- 172.21.15.47:0/4183460825 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd55a890 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.135+0000 7fdbde386b80 1 --2- 172.21.15.47:0/4183460825 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 0x5638dd553eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.135+0000 7fdbde386b80 1 --2- 172.21.15.47:0/4183460825 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd54e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.135+0000 7fdbde386b80 1 -- 172.21.15.47:0/4183460825 >> 172.21.15.47:0/4183460825 conn(0x5638dd47ae60 msgr2=0x5638dd548990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.151+0000 7fdbde386b80 1 -- 172.21.15.47:0/4183460825 shutdown_connections 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.151+0000 7fdbde386b80 1 -- 172.21.15.47:0/4183460825 wait complete. 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.151+0000 7fdbde386b80 1 -- start start 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.152+0000 7fdbde386b80 1 Processor -- start 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.152+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd55a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.152+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd557920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.153+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 0x5638dd557e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.153+0000 7fdbde386b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5638dd55add0 con 0x5638dd553ad0 2024-05-11T12:34:25.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.153+0000 7fdbde386b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5638dd406c30 con 0x5638dd54ea60 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.153+0000 7fdbde386b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5638dd55af50 con 0x5638dd54e0a0 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.153+0000 7fdbde386b80 10 client.0 _ll_register_callbacks cb 0x5638dd438920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.158+0000 7fdbd108b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd557920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.158+0000 7fdbd188c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd55a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.158+0000 7fdbd188c700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd55a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40520/0 (socket says 172.21.15.47:40520) 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.158+0000 7fdbd188c700 1 -- 172.21.15.47:0/2738443799 learned_addr learned my addr 172.21.15.47:0/2738443799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.160+0000 7fdbd208d700 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 0x5638dd557e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.160+0000 7fdbd188c700 1 -- 172.21.15.47:0/2738443799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 msgr2=0x5638dd557920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.160+0000 7fdbd188c700 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd557920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.160+0000 7fdbd188c700 1 -- 172.21.15.47:0/2738443799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 msgr2=0x5638dd557e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.160+0000 7fdbd188c700 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 0x5638dd557e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.160+0000 7fdbd188c700 1 -- 172.21.15.47:0/2738443799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd43c690 con 0x5638dd54e0a0 2024-05-11T12:34:25.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.165+0000 7fdbd188c700 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd55a530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4018b70 tx=0x7fdbc4004af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64565d9c32df9242 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:25.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.166+0000 7fdbb57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:34:25.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.170+0000 7fdbb57fa700 1 -- 172.21.15.47:0/2738443799 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40099a0 con 0x5638dd54e0a0 2024-05-11T12:34:25.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.170+0000 7fdbb57fa700 1 -- 172.21.15.47:0/2738443799 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fdbc4009b40 con 0x5638dd54e0a0 2024-05-11T12:34:25.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.170+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbc80050d0 con 0x5638dd54e0a0 2024-05-11T12:34:25.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.176+0000 7fdbb57fa700 1 -- 172.21.15.47:0/2738443799 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdbc4009d40 con 0x5638dd54e0a0 2024-05-11T12:34:25.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.176+0000 7fdbde386b80 10 client.25450 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:25.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.176+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5638dd587520 con 0x5638dd54e0a0 2024-05-11T12:34:25.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.180+0000 7fdbb57fa700 1 -- 172.21.15.47:0/2738443799 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbc80050d0 con 0x5638dd54e0a0 2024-05-11T12:34:25.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.181+0000 7fdbb57fa700 1 -- 172.21.15.47:0/2738443799 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fdbc4016520 con 0x5638dd54e0a0 2024-05-11T12:34:25.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.181+0000 7fdbde386b80 10 client.25450 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.181+0000 7fdbde386b80 10 client.25450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.182+0000 7fdbde386b80 20 client.25450 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.182+0000 7fdbde386b80 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd589b60 0x5638dd58bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.182+0000 7fdbde386b80 4 client.25450 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:34:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.182+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638dd4ae240 con 0x5638dd589b60 2024-05-11T12:34:25.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.189+0000 7fdbd108b700 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd589b60 0x5638dd58bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.195+0000 7fdbd108b700 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd589b60 0x5638dd58bf50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467a80af0f440e7a server_cookie=16dfc32b9d8e0630 in_seq=0 out_seq=0 2024-05-11T12:34:25.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.195+0000 7fdbb57fa700 10 client.25450 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:25.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.202+0000 7fdbb57fa700 1 -- 172.21.15.47:0/2738443799 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5638dd4ae240 con 0x5638dd589b60 2024-05-11T12:34:25.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.202+0000 7fdbb57fa700 10 client.25450 handle_command_reply: tid=0 2024-05-11T12:34:25.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.202+0000 7fdbde386b80 1 client.25450 shutdown 2024-05-11T12:34:25.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.202+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 msgr2=0x5638dd55a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.202+0000 7fdbde386b80 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd55a530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4018b70 tx=0x7fdbc4004af0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.203+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 shutdown_connections 2024-05-11T12:34:25.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.203+0000 7fdbde386b80 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd589b60 0x5638dd58bf50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.203+0000 7fdbde386b80 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd553ad0 0x5638dd557e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.203+0000 7fdbde386b80 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd54e0a0 0x5638dd55a530 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.203+0000 7fdbde386b80 1 --2- 172.21.15.47:0/2738443799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd557920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.203+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 >> 172.21.15.47:0/2738443799 conn(0x5638dd47ae60 msgr2=0x5638dd543460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:25.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.203+0000 7fdbb57fa700 0 client.25450 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.210+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 shutdown_connections 2024-05-11T12:34:25.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.210+0000 7fdbde386b80 1 -- 172.21.15.47:0/2738443799 wait complete. 2024-05-11T12:34:25.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.210+0000 7fdbde386b80 20 client.25450 trim_cache size 0 max 16384 2024-05-11T12:34:25.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.261+0000 7fdbde386b80 1 Processor -- start 2024-05-11T12:34:25.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.261+0000 7fdbde386b80 1 -- start start 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.262+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd54e0a0 0x5638dd553180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd54d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd553ad0 0x5638dd54d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbde386b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb000b710 con 0x5638dd54e0a0 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbde386b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb000dee0 con 0x5638dd54ea60 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbde386b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb000bcd0 con 0x5638dd553ad0 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbd108b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd54d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbd108b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd54d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:36252/0 (socket says 172.21.15.47:36252) 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.263+0000 7fdbd108b700 1 -- 172.21.15.47:0/1591878088 learned_addr learned my addr 172.21.15.47:0/1591878088 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:25.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.268+0000 7fdbd188c700 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd54e0a0 0x5638dd553180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.269+0000 7fdbd208d700 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd553ad0 0x5638dd54d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.273+0000 7fdbd108b700 1 -- 172.21.15.47:0/1591878088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd553ad0 msgr2=0x5638dd54d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.273+0000 7fdbd108b700 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd553ad0 0x5638dd54d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.273+0000 7fdbd108b700 1 -- 172.21.15.47:0/1591878088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd54e0a0 msgr2=0x5638dd553180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.273+0000 7fdbd108b700 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd54e0a0 0x5638dd553180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.273+0000 7fdbd108b700 1 -- 172.21.15.47:0/1591878088 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd545590 con 0x5638dd54ea60 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.274+0000 7fdbd108b700 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd54d090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc006f60 tx=0x7fdbbc006ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0dfd69ceb89c720 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.275+0000 7fdbb77fe700 1 -- 172.21.15.47:0/1591878088 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0ae290 con 0x5638dd54ea60 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.276+0000 7fdbb77fe700 1 -- 172.21.15.47:0/1591878088 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbbc0b2d40 con 0x5638dd54ea60 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.277+0000 7fdbde386b80 1 -- 172.21.15.47:0/1591878088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 msgr2=0x5638dd54d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.277+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd54d090 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc006f60 tx=0x7fdbbc006ed0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.277+0000 7fdbde386b80 1 -- 172.21.15.47:0/1591878088 shutdown_connections 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.277+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd553ad0 0x5638dd54d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.277+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd54e0a0 0x5638dd553180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.277+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1591878088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd54ea60 0x5638dd54d090 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.277+0000 7fdbde386b80 1 -- 172.21.15.47:0/1591878088 >> 172.21.15.47:0/1591878088 conn(0x5638dd47ae60 msgr2=0x5638dd552a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.278+0000 7fdbde386b80 1 -- 172.21.15.47:0/1591878088 shutdown_connections 2024-05-11T12:34:25.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.279+0000 7fdbde386b80 1 -- 172.21.15.47:0/1591878088 wait complete. 2024-05-11T12:34:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.279+0000 7fdbde386b80 1 -- start start 2024-05-11T12:34:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.280+0000 7fdbde386b80 1 Processor -- start 2024-05-11T12:34:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.281+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd553ad0 0x5638dd560240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.281+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd560780 0x5638dd590810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.282+0000 7fdbde386b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd590d50 0x5638dd5911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.282+0000 7fdbde386b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb000c3a0 con 0x5638dd590d50 2024-05-11T12:34:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.282+0000 7fdbde386b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb000d810 con 0x5638dd553ad0 2024-05-11T12:34:25.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.282+0000 7fdbde386b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb000ca70 con 0x5638dd560780 2024-05-11T12:34:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.282+0000 7fdbde386b80 10 client.0 _ll_register_callbacks cb 0x5638dd438920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.283+0000 7fdbd108b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd560780 0x5638dd590810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.283+0000 7fdbd108b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd560780 0x5638dd590810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:40530/0 (socket says 172.21.15.47:40530) 2024-05-11T12:34:25.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.283+0000 7fdbd108b700 1 -- 172.21.15.47:0/1718903569 learned_addr learned my addr 172.21.15.47:0/1718903569 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.287+0000 7fdbd108b700 1 -- 172.21.15.47:0/1718903569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd553ad0 msgr2=0x5638dd560240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.287+0000 7fdbd108b700 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd553ad0 0x5638dd560240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.287+0000 7fdbd108b700 1 -- 172.21.15.47:0/1718903569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd590d50 msgr2=0x5638dd5911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.288+0000 7fdbd108b700 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd590d50 0x5638dd5911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.288+0000 7fdbd108b700 1 -- 172.21.15.47:0/1718903569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd560c30 con 0x5638dd560780 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.288+0000 7fdbd108b700 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd560780 0x5638dd590810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0a7150 tx=0x7fdbbc0b2ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b92dc8fff519e523 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.288+0000 7fdbb57fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.289+0000 7fdbb57fa700 1 -- 172.21.15.47:0/1718903569 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0a9650 con 0x5638dd560780 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.289+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5638dd4a82e0 con 0x5638dd560780 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.289+0000 7fdbb57fa700 1 -- 172.21.15.47:0/1718903569 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fdbbc011a40 con 0x5638dd560780 2024-05-11T12:34:25.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.289+0000 7fdbb57fa700 1 -- 172.21.15.47:0/1718903569 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fdbbc0b2d40 con 0x5638dd560780 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.290+0000 7fdbde386b80 10 client.25456 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.290+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5638dd5f44d0 con 0x5638dd560780 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.291+0000 7fdbb57fa700 1 -- 172.21.15.47:0/1718903569 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbbc0ae290 con 0x5638dd560780 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.291+0000 7fdbb57fa700 1 -- 172.21.15.47:0/1718903569 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fdbbc0a8d30 con 0x5638dd560780 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.292+0000 7fdbde386b80 10 client.25456 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.292+0000 7fdbde386b80 10 client.25456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.292+0000 7fdbde386b80 20 client.25456 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.292+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd5f4ad0 0x5638dd5f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.292+0000 7fdbde386b80 4 client.25456 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:34:25.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.292+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638dd4a82e0 con 0x5638dd5f4ad0 2024-05-11T12:34:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.297+0000 7fdbd188c700 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd5f4ad0 0x5638dd5f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.298+0000 7fdbb57fa700 10 client.25456 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:25.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.298+0000 7fdbd188c700 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd5f4ad0 0x5638dd5f6ef0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e186fe194767c8 server_cookie=908aa5753ac112a0 in_seq=0 out_seq=0 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.300+0000 7fdbb57fa700 1 -- 172.21.15.47:0/1718903569 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638dd4a82e0 con 0x5638dd5f4ad0 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.300+0000 7fdbb57fa700 10 client.25456 handle_command_reply: tid=0 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.300+0000 7fdbde386b80 1 client.25456 shutdown 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.300+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd560780 msgr2=0x5638dd590810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.300+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd560780 0x5638dd590810 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0a7150 tx=0x7fdbbc0b2ac0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 shutdown_connections 2024-05-11T12:34:25.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5638dd5f4ad0 0x5638dd5f6ef0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5638dd590d50 0x5638dd5911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5638dd553ad0 0x5638dd560240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 --2- 172.21.15.47:0/1718903569 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5638dd560780 0x5638dd590810 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 >> 172.21.15.47:0/1718903569 conn(0x5638dd47ae60 msgr2=0x5638dd54d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbb57fa700 0 client.25456 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 shutdown_connections 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 1 -- 172.21.15.47:0/1718903569 wait complete. 2024-05-11T12:34:25.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.301+0000 7fdbde386b80 20 client.25456 trim_cache size 0 max 16384 2024-05-11T12:34:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdbb80b2a90 msgr2=0x7fdbb80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdbb80b2a90 0x7fdbb80b4f50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc1047e0 tx=0x7fdbc4009250 comp rx=0 tx=0).stop 2024-05-11T12:34:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc103fa0 msgr2=0x7fdbcc1a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc1a0f80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00b530 tx=0x7fdbbc00b8f0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 shutdown_connections 2024-05-11T12:34:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fdbcc108510 0x7fdbcc10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdbcc103600 0x7fdbcc1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdbb80b2a90 0x7fdbb80b4f50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 --2- 172.21.15.47:0/3419967248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdbcc103fa0 0x7fdbcc1a0f80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:25.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.306+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 >> 172.21.15.47:0/3419967248 conn(0x7fdbcc07a6b0 msgr2=0x7fdbcc101470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:25.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.307+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 shutdown_connections 2024-05-11T12:34:25.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:25.307+0000 7fdbd3af0700 1 -- 172.21.15.47:0/3419967248 wait complete. 2024-05-11T12:34:25.781 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:25 smithi047 ceph-mon[30375]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 679 MiB/s wr, 1.87k op/s 2024-05-11T12:34:25.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:25 smithi064 ceph-mon[34537]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 679 MiB/s wr, 1.87k op/s 2024-05-11T12:34:25.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:25 smithi050 ceph-mon[34072]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 679 MiB/s wr, 1.87k op/s 2024-05-11T12:34:27.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:27 smithi047 ceph-mon[30375]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 809 MiB/s wr, 2.14k op/s 2024-05-11T12:34:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:27 smithi064 ceph-mon[34537]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 809 MiB/s wr, 2.14k op/s 2024-05-11T12:34:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:27 smithi050 ceph-mon[34072]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 809 MiB/s wr, 2.14k op/s 2024-05-11T12:34:28.728 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:34:28.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:34:29.658 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:34:29.750 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:29 smithi050 ceph-mon[34072]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 MiB/s wr, 1.76k op/s 2024-05-11T12:34:29.790 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:29 smithi047 ceph-mon[30375]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 MiB/s wr, 1.76k op/s 2024-05-11T12:34:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:29 smithi064 ceph-mon[34537]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 784 MiB/s wr, 1.76k op/s 2024-05-11T12:34:31.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:31 smithi064 ceph-mon[34537]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 777 MiB/s wr, 1.70k op/s 2024-05-11T12:34:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:31 smithi050 ceph-mon[34072]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 777 MiB/s wr, 1.70k op/s 2024-05-11T12:34:32.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:31 smithi047 ceph-mon[30375]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 777 MiB/s wr, 1.70k op/s 2024-05-11T12:34:33.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:33 smithi064 ceph-mon[34537]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 836 MiB/s wr, 2.02k op/s 2024-05-11T12:34:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:33 smithi050 ceph-mon[34072]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 836 MiB/s wr, 2.02k op/s 2024-05-11T12:34:33.953 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:33 smithi047 ceph-mon[30375]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 836 MiB/s wr, 2.02k op/s 2024-05-11T12:34:34.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.544+0000 7f49724ef700 1 -- 172.21.15.47:0/2988562152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c11cd30 msgr2=0x7f496c11f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:34.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.544+0000 7f49724ef700 1 --2- 172.21.15.47:0/2988562152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c11cd30 0x7f496c11f120 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4964009f90 tx=0x7f496400b330 comp rx=0 tx=0).stop 2024-05-11T12:34:34.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.544+0000 7f49724ef700 1 -- 172.21.15.47:0/2988562152 shutdown_connections 2024-05-11T12:34:34.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.544+0000 7f49724ef700 1 --2- 172.21.15.47:0/2988562152 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c11cd30 0x7f496c11f120 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.544+0000 7f49724ef700 1 --2- 172.21.15.47:0/2988562152 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f496c111c40 0x7f496c10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.544+0000 7f49724ef700 1 --2- 172.21.15.47:0/2988562152 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c1120b0 0x7f496c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.544+0000 7f49724ef700 1 -- 172.21.15.47:0/2988562152 >> 172.21.15.47:0/2988562152 conn(0x7f496c06e530 msgr2=0x7f496c06e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:34.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.546+0000 7f49724ef700 1 -- 172.21.15.47:0/2988562152 shutdown_connections 2024-05-11T12:34:34.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.546+0000 7f49724ef700 1 -- 172.21.15.47:0/2988562152 wait complete. 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.547+0000 7f49724ef700 1 Processor -- start 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.547+0000 7f49724ef700 1 -- start start 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.547+0000 7f49724ef700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f496c111c40 0x7f496c1ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.547+0000 7f49724ef700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c1120b0 0x7f496c1aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.548+0000 7f49724ef700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c11cd30 0x7f496c1a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.548+0000 7f49724ef700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f496c122520 con 0x7f496c11cd30 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.548+0000 7f49724ef700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1223a0 con 0x7f496c111c40 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.548+0000 7f49724ef700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1226a0 con 0x7f496c1120b0 2024-05-11T12:34:34.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.548+0000 7f4971cee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c11cd30 0x7f496c1a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.548+0000 7f4971cee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c11cd30 0x7f496c1a87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:49442/0 (socket says 172.21.15.47:49442) 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.549+0000 7f4971cee700 1 -- 172.21.15.47:0/1014910681 learned_addr learned my addr 172.21.15.47:0/1014910681 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.549+0000 7f4971cee700 1 -- 172.21.15.47:0/1014910681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c1120b0 msgr2=0x7f496c1aeb80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.549+0000 7f4971cee700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c1120b0 0x7f496c1aeb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.549+0000 7f4971cee700 1 -- 172.21.15.47:0/1014910681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f496c111c40 msgr2=0x7f496c1ae640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.549+0000 7f4971cee700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f496c111c40 0x7f496c1ae640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.549+0000 7f4971cee700 1 -- 172.21.15.47:0/1014910681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c1a8fa0 con 0x7f496c11cd30 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.550+0000 7f4971cee700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c11cd30 0x7f496c1a87c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4964009f60 tx=0x7f496400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.550+0000 7f49627fc700 1 -- 172.21.15.47:0/1014910681 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964043070 con 0x7f496c11cd30 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.551+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c1a9230 con 0x7f496c11cd30 2024-05-11T12:34:34.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.551+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c075780 con 0x7f496c11cd30 2024-05-11T12:34:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.559+0000 7f497dfe7b80 1 Processor -- start 2024-05-11T12:34:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.559+0000 7f497dfe7b80 1 -- start start 2024-05-11T12:34:34.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.560+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.560+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a024fcc20 0x564a02500890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.560+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 0x564a02501210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.560+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564a0244e8e0 con 0x564a02500dd0 2024-05-11T12:34:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.560+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564a023ed840 con 0x564a024fc300 2024-05-11T12:34:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.560+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a0244a8a0 con 0x564a024fcc20 2024-05-11T12:34:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.560+0000 7f49627fc700 1 -- 172.21.15.47:0/1014910681 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4964007b40 con 0x7f496c11cd30 2024-05-11T12:34:34.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.561+0000 7f49627fc700 1 -- 172.21.15.47:0/1014910681 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49640052a0 con 0x7f496c11cd30 2024-05-11T12:34:34.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.565+0000 7f49714ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a024fcc20 0x564a02500890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.565+0000 7f4970cec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.567+0000 7f4970cec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024fc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:41938/0 (socket says 172.21.15.47:41938) 2024-05-11T12:34:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.567+0000 7f49714ed700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a024fcc20 0x564a02500890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:53074/0 (socket says 172.21.15.47:53074) 2024-05-11T12:34:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.567+0000 7f49714ed700 1 -- 172.21.15.47:0/4129738990 learned_addr learned my addr 172.21.15.47:0/4129738990 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:34.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.567+0000 7f49627fc700 1 -- 172.21.15.47:0/1014910681 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4964046070 con 0x7f496c11cd30 2024-05-11T12:34:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.568+0000 7f49714ed700 1 -- 172.21.15.47:0/4129738990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 msgr2=0x564a024fc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.568+0000 7f49714ed700 1 --2- 172.21.15.47:0/4129738990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024fc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.568+0000 7f49714ed700 1 -- 172.21.15.47:0/4129738990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 msgr2=0x564a02501210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.568+0000 7f49714ed700 1 --2- 172.21.15.47:0/4129738990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 0x564a02501210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.568+0000 7f49714ed700 1 -- 172.21.15.47:0/4129738990 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a0239b950 con 0x564a024fcc20 2024-05-11T12:34:34.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.569+0000 7f49627fc700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f49580b28e0 0x7f49580b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.570+0000 7f49627fc700 1 -- 172.21.15.47:0/1014910681 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4964105330 con 0x7f496c11cd30 2024-05-11T12:34:34.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.571+0000 7f4970cec700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f49580b28e0 0x7f49580b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.572+0000 7f49714ed700 1 --2- 172.21.15.47:0/4129738990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a024fcc20 0x564a02500890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4968009b50 tx=0x7f4968009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6bfa07ee28930f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:34.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.578+0000 7f49577fe700 1 -- 172.21.15.47:0/4129738990 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496800eb00 con 0x564a024fcc20 2024-05-11T12:34:34.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.578+0000 7f49577fe700 1 -- 172.21.15.47:0/4129738990 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4968004ce0 con 0x564a024fcc20 2024-05-11T12:34:34.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.579+0000 7f497dfe7b80 1 -- 172.21.15.47:0/4129738990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a024fcc20 msgr2=0x564a02500890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:34.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/4129738990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a024fcc20 0x564a02500890 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f4968009b50 tx=0x7f4968009e60 comp rx=0 tx=0).stop 2024-05-11T12:34:34.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 -- 172.21.15.47:0/4129738990 shutdown_connections 2024-05-11T12:34:34.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/4129738990 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 0x564a02501210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/4129738990 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024fc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/4129738990 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a024fcc20 0x564a02500890 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 -- 172.21.15.47:0/4129738990 >> 172.21.15.47:0/4129738990 conn(0x564a02428ee0 msgr2=0x564a024f6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 -- 172.21.15.47:0/4129738990 shutdown_connections 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 -- 172.21.15.47:0/4129738990 wait complete. 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.580+0000 7f497dfe7b80 1 -- start start 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.581+0000 7f497dfe7b80 1 Processor -- start 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.581+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a02508010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.581+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 0x564a02508550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.581+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02505360 0x564a025057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.582+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564a02509080 con 0x564a02500dd0 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.582+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564a023b4cb0 con 0x564a024fc300 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.582+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564a02509200 con 0x564a02505360 2024-05-11T12:34:34.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.582+0000 7f497dfe7b80 10 client.0 _ll_register_callbacks cb 0x564a023e69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.585+0000 7f49714ed700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a02508010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.585+0000 7f49714ed700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a02508010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:41940/0 (socket says 172.21.15.47:41940) 2024-05-11T12:34:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.585+0000 7f49714ed700 1 -- 172.21.15.47:0/3311133306 learned_addr learned my addr 172.21.15.47:0/3311133306 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:34.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.585+0000 7f4971cee700 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02505360 0x564a025057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.585+0000 7f4970cec700 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 0x564a02508550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.589+0000 7f49714ed700 1 -- 172.21.15.47:0/3311133306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02505360 msgr2=0x564a025057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.589+0000 7f49714ed700 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02505360 0x564a025057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.589+0000 7f49714ed700 1 -- 172.21.15.47:0/3311133306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 msgr2=0x564a02508550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.589+0000 7f49714ed700 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 0x564a02508550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.589+0000 7f49714ed700 1 -- 172.21.15.47:0/3311133306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a023ea710 con 0x564a024fc300 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.593+0000 7f4970cec700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f49580b28e0 0x7f49580b4da0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f496c1108e0 tx=0x7f495c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.593+0000 7f49714ed700 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a02508010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f49680056c0 tx=0x7f49680056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fceb1e8d6c9fa38 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.593+0000 7f4954ff9700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.594+0000 7f4954ff9700 1 -- 172.21.15.47:0/3311133306 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4968027060 con 0x564a024fc300 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.594+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a0245c2c0 con 0x564a024fc300 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.595+0000 7f4954ff9700 1 -- 172.21.15.47:0/3311133306 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f496800ead0 con 0x564a024fc300 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.595+0000 7f4954ff9700 1 -- 172.21.15.47:0/3311133306 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f496800ecd0 con 0x564a024fc300 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.597+0000 7f497dfe7b80 10 client.25445 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:34.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.597+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564a02524bf0 con 0x564a024fc300 2024-05-11T12:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f4954ff9700 1 -- 172.21.15.47:0/3311133306 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564a0245c2c0 con 0x564a024fc300 2024-05-11T12:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f4954ff9700 1 -- 172.21.15.47:0/3311133306 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4968003a30 con 0x564a024fc300 2024-05-11T12:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f497dfe7b80 10 client.25445 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f497dfe7b80 10 client.25445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f497dfe7b80 20 client.25445 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02527580 0x564a025299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f497dfe7b80 4 client.25445 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:34:34.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.603+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a02456360 con 0x564a02527580 2024-05-11T12:34:34.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.605+0000 7f4970cec700 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02527580 0x564a025299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.607+0000 7f4970cec700 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02527580 0x564a025299c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9c80c8452443e5 server_cookie=d30e1ea0bbc4a685 in_seq=0 out_seq=0 2024-05-11T12:34:34.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.607+0000 7f4954ff9700 10 client.25445 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:34.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.614+0000 7f4954ff9700 1 -- 172.21.15.47:0/3311133306 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564a02456360 con 0x564a02527580 2024-05-11T12:34:34.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.614+0000 7f4954ff9700 10 client.25445 handle_command_reply: tid=0 2024-05-11T12:34:34.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.614+0000 7f497dfe7b80 1 client.25445 shutdown 2024-05-11T12:34:34.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.614+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 msgr2=0x564a02508010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.614+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a02508010 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f49680056c0 tx=0x7f49680056f0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 shutdown_connections 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02527580 0x564a025299c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02505360 0x564a025057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a02508010 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/3311133306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02500dd0 0x564a02508550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 >> 172.21.15.47:0/3311133306 conn(0x564a02428ee0 msgr2=0x564a024fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f4954ff9700 0 client.25445 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 shutdown_connections 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 1 -- 172.21.15.47:0/3311133306 wait complete. 2024-05-11T12:34:34.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.617+0000 7f497dfe7b80 20 client.25445 trim_cache size 0 max 16384 2024-05-11T12:34:34.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.674+0000 7f497dfe7b80 1 Processor -- start 2024-05-11T12:34:34.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.674+0000 7f497dfe7b80 1 -- start start 2024-05-11T12:34:34.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.675+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024ffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.675+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02500dd0 0x564a02523280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.675+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02505360 0x564a0252f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.676+0000 7f49714ed700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024ffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.676+0000 7f49714ed700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024ffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:41954/0 (socket says 172.21.15.47:41954) 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.676+0000 7f49714ed700 1 -- 172.21.15.47:0/2002230087 learned_addr learned my addr 172.21.15.47:0/2002230087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.677+0000 7f4970cec700 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02500dd0 0x564a02523280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.677+0000 7f4971cee700 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02505360 0x564a0252f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.677+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f495000b770 con 0x564a02505360 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.677+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f495000dff0 con 0x564a024fc300 2024-05-11T12:34:34.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.678+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f495000bde0 con 0x564a02500dd0 2024-05-11T12:34:34.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.679+0000 7f4971cee700 1 -- 172.21.15.47:0/2002230087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02500dd0 msgr2=0x564a02523280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.679+0000 7f4971cee700 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02500dd0 0x564a02523280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.679+0000 7f4971cee700 1 -- 172.21.15.47:0/2002230087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 msgr2=0x564a024ffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.679+0000 7f4971cee700 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024ffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.679+0000 7f4971cee700 1 -- 172.21.15.47:0/2002230087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a02500c00 con 0x564a02505360 2024-05-11T12:34:34.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.680+0000 7f4971cee700 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02505360 0x564a0252f290 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f496403ffd0 tx=0x7f496400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d4d6a76360ddb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:34.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.681+0000 7f4956ffd700 1 -- 172.21.15.47:0/2002230087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496404d030 con 0x564a02505360 2024-05-11T12:34:34.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.682+0000 7f4956ffd700 1 -- 172.21.15.47:0/2002230087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49640ccfb0 con 0x564a02505360 2024-05-11T12:34:34.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.683+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02505360 msgr2=0x564a0252f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:34.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.683+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02505360 0x564a0252f290 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f496403ffd0 tx=0x7f496400b330 comp rx=0 tx=0).stop 2024-05-11T12:34:34.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.684+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 shutdown_connections 2024-05-11T12:34:34.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.684+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a02505360 0x564a0252f290 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.684+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a024fc300 0x564a024ffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.684+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/2002230087 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a02500dd0 0x564a02523280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.684+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 >> 172.21.15.47:0/2002230087 conn(0x564a02428ee0 msgr2=0x564a024ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:34.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.688+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 shutdown_connections 2024-05-11T12:34:34.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.688+0000 7f497dfe7b80 1 -- 172.21.15.47:0/2002230087 wait complete. 2024-05-11T12:34:34.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.689+0000 7f497dfe7b80 1 -- start start 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.689+0000 7f497dfe7b80 1 Processor -- start 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.690+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a024fc300 0x564a0251f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.691+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a02500dd0 0x564a0251fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.691+0000 7f497dfe7b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a0251c5d0 0x564a0251ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.691+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f495000c4b0 con 0x564a024fc300 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.691+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f495000d920 con 0x564a02500dd0 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.691+0000 7f497dfe7b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f495000cb80 con 0x564a0251c5d0 2024-05-11T12:34:34.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.691+0000 7f497dfe7b80 10 client.0 _ll_register_callbacks cb 0x564a023e69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.692+0000 7f4970cec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a02500dd0 0x564a0251fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.693+0000 7f4970cec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a02500dd0 0x564a0251fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:41968/0 (socket says 172.21.15.47:41968) 2024-05-11T12:34:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.693+0000 7f4970cec700 1 -- 172.21.15.47:0/945260428 learned_addr learned my addr 172.21.15.47:0/945260428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:34.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.693+0000 7f4971cee700 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a0251c5d0 0x564a0251ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.697+0000 7f49714ed700 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a024fc300 0x564a0251f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.697+0000 7f4970cec700 1 -- 172.21.15.47:0/945260428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a0251c5d0 msgr2=0x564a0251ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.697+0000 7f4970cec700 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a0251c5d0 0x564a0251ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.697+0000 7f4970cec700 1 -- 172.21.15.47:0/945260428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a024fc300 msgr2=0x564a0251f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.697+0000 7f4970cec700 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a024fc300 0x564a0251f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.697+0000 7f4970cec700 1 -- 172.21.15.47:0/945260428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a025237c0 con 0x564a02500dd0 2024-05-11T12:34:34.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.702+0000 7f4970cec700 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a02500dd0 0x564a0251fb40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b010 tx=0x7f495c000fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f616a114b71fc2d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:34.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.702+0000 7f4954ff9700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:34:34.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.705+0000 7f4954ff9700 1 -- 172.21.15.47:0/945260428 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c008c20 con 0x564a02500dd0 2024-05-11T12:34:34.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.705+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564a022ce2d0 con 0x564a02500dd0 2024-05-11T12:34:34.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.705+0000 7f4954ff9700 1 -- 172.21.15.47:0/945260428 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f495c008dc0 con 0x564a02500dd0 2024-05-11T12:34:34.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.707+0000 7f4954ff9700 1 -- 172.21.15.47:0/945260428 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f495c009050 con 0x564a02500dd0 2024-05-11T12:34:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.708+0000 7f497dfe7b80 10 client.25451 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:34.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.708+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564a02592a30 con 0x564a02500dd0 2024-05-11T12:34:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f4954ff9700 1 -- 172.21.15.47:0/945260428 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564a022ce2d0 con 0x564a02500dd0 2024-05-11T12:34:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f4954ff9700 1 -- 172.21.15.47:0/945260428 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f495c001630 con 0x564a02500dd0 2024-05-11T12:34:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f497dfe7b80 10 client.25451 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f497dfe7b80 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f497dfe7b80 20 client.25451 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02593120 0x564a02595540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f497dfe7b80 4 client.25451 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:34:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.711+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564a0241eda0 con 0x564a02593120 2024-05-11T12:34:34.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.717+0000 7f49714ed700 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02593120 0x564a02595540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:34.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.723+0000 7f49714ed700 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02593120 0x564a02595540 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95eae507891b6ffd server_cookie=eb4bb48a41489161 in_seq=0 out_seq=0 2024-05-11T12:34:34.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.723+0000 7f4954ff9700 10 client.25451 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:34.961 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:34:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:34:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "82a15458-69a9-4f61-b402-4f583d0f5d2b", 2024-05-11T12:34:34.961 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:34:34.961 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:34:34.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.955+0000 7f4954ff9700 1 -- 172.21.15.47:0/945260428 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564a0241eda0 con 0x564a02593120 2024-05-11T12:34:34.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.955+0000 7f4954ff9700 10 client.25451 handle_command_reply: tid=0 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.955+0000 7f497dfe7b80 1 client.25451 shutdown 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a02500dd0 msgr2=0x564a0251fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a02500dd0 0x564a0251fb40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b010 tx=0x7f495c000fd0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 shutdown_connections 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564a02593120 0x564a02595540 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564a0251c5d0 0x564a0251ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564a024fc300 0x564a0251f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 --2- 172.21.15.47:0/945260428 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564a02500dd0 0x564a0251fb40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.956+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 >> 172.21.15.47:0/945260428 conn(0x564a02428ee0 msgr2=0x564a025037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.957+0000 7f4954ff9700 0 client.25451 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.957+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 shutdown_connections 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.957+0000 7f497dfe7b80 1 -- 172.21.15.47:0/945260428 wait complete. 2024-05-11T12:34:34.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.958+0000 7f497dfe7b80 20 client.25451 trim_cache size 0 max 16384 2024-05-11T12:34:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.960+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f49580b28e0 msgr2=0x7f49580b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.960+0000 7f49724ef700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f49580b28e0 0x7f49580b4da0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f496c1108e0 tx=0x7f495c00b040 comp rx=0 tx=0).stop 2024-05-11T12:34:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.960+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c11cd30 msgr2=0x7f496c1a87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:34.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.960+0000 7f49724ef700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c11cd30 0x7f496c1a87c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4964009f60 tx=0x7f496400ce20 comp rx=0 tx=0).stop 2024-05-11T12:34:34.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.964+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 shutdown_connections 2024-05-11T12:34:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.965+0000 7f49724ef700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f496c11cd30 0x7f496c1a87c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.965+0000 7f49724ef700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f49580b28e0 0x7f49580b4da0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.965+0000 7f49724ef700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f496c111c40 0x7f496c1ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.965+0000 7f49724ef700 1 --2- 172.21.15.47:0/1014910681 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f496c1120b0 0x7f496c1aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.965+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 >> 172.21.15.47:0/1014910681 conn(0x7f496c06e530 msgr2=0x7f496c10f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.965+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 shutdown_connections 2024-05-11T12:34:34.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:34.965+0000 7f49724ef700 1 -- 172.21.15.47:0/1014910681 wait complete. 2024-05-11T12:34:35.846 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:35 smithi047 ceph-mon[30375]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 692 MiB/s wr, 1.72k op/s 2024-05-11T12:34:35.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:35 smithi064 ceph-mon[34537]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 692 MiB/s wr, 1.72k op/s 2024-05-11T12:34:35.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:35 smithi050 ceph-mon[34072]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 692 MiB/s wr, 1.72k op/s 2024-05-11T12:34:36.606 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:34:36.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:36 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:34:36.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:36 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:36.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:36 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:34:36.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:36 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:34:36.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:36 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:36.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:36 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:34:37.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:36 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:34:37.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:36 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:37.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:36 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:34:37.708 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:34:37.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:37 smithi050 ceph-mon[34072]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 1.86k op/s 2024-05-11T12:34:38.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:37 smithi047 ceph-mon[30375]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 1.86k op/s 2024-05-11T12:34:38.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:37 smithi064 ceph-mon[34537]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 836 MiB/s wr, 1.86k op/s 2024-05-11T12:34:39.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:39 smithi050 ceph-mon[34072]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 806 MiB/s wr, 1.65k op/s 2024-05-11T12:34:40.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:39 smithi047 ceph-mon[30375]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 806 MiB/s wr, 1.65k op/s 2024-05-11T12:34:40.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:39 smithi064 ceph-mon[34537]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 806 MiB/s wr, 1.65k op/s 2024-05-11T12:34:41.779 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:41 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:41.779 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:41 smithi047 ceph-mon[30375]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 711 MiB/s wr, 1.60k op/s 2024-05-11T12:34:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:41 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:41 smithi050 ceph-mon[34072]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 711 MiB/s wr, 1.60k op/s 2024-05-11T12:34:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:41 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:34:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:41 smithi064 ceph-mon[34537]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 711 MiB/s wr, 1.60k op/s 2024-05-11T12:34:43.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.610+0000 7f1acff3b700 1 -- 172.21.15.47:0/3355921109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ac8108470 msgr2=0x7f1ac806baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.610+0000 7f1acff3b700 1 --2- 172.21.15.47:0/3355921109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ac8108470 0x7f1ac806baf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40098f0 tx=0x7f1ac4009c00 comp rx=0 tx=0).stop 2024-05-11T12:34:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.612+0000 7f1acff3b700 1 -- 172.21.15.47:0/3355921109 shutdown_connections 2024-05-11T12:34:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.612+0000 7f1acff3b700 1 --2- 172.21.15.47:0/3355921109 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8105f80 0x7f1ac8106410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.612+0000 7f1acff3b700 1 --2- 172.21.15.47:0/3355921109 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ac8108470 0x7f1ac806baf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.612+0000 7f1acff3b700 1 --2- 172.21.15.47:0/3355921109 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 0x7f1ac806c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.612+0000 7f1acff3b700 1 -- 172.21.15.47:0/3355921109 >> 172.21.15.47:0/3355921109 conn(0x7f1ac80793c0 msgr2=0x7f1ac80797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:43.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.612+0000 7f1acff3b700 1 -- 172.21.15.47:0/3355921109 shutdown_connections 2024-05-11T12:34:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.612+0000 7f1acff3b700 1 -- 172.21.15.47:0/3355921109 wait complete. 2024-05-11T12:34:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 Processor -- start 2024-05-11T12:34:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 -- start start 2024-05-11T12:34:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 0x7f1ac8074010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ac8105f80 0x7f1ac8074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8108470 0x7f1ac810ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810b280 con 0x7f1ac8105f80 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810b100 con 0x7f1ac806c030 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acff3b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810b400 con 0x7f1ac8108470 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acdcd7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 0x7f1ac8074010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acdcd7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 0x7f1ac8074010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:35564/0 (socket says 172.21.15.47:35564) 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.614+0000 7f1acdcd7700 1 -- 172.21.15.47:0/180483712 learned_addr learned my addr 172.21.15.47:0/180483712 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.615+0000 7f1ace4d8700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8108470 0x7f1ac810ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.615+0000 7f1acdcd7700 1 -- 172.21.15.47:0/180483712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8108470 msgr2=0x7f1ac810ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.615+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8108470 0x7f1ac810ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.615+0000 7f1acdcd7700 1 -- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ac8105f80 msgr2=0x7f1ac8074550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.615+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ac8105f80 0x7f1ac8074550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.615+0000 7f1acdcd7700 1 -- 172.21.15.47:0/180483712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81170c0 con 0x7f1ac806c030 2024-05-11T12:34:43.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.618+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 0x7f1ac8074010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000c00 tx=0x7f1ac400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:43.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.623+0000 7f1abeffd700 1 -- 172.21.15.47:0/180483712 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400bca0 con 0x7f1ac806c030 2024-05-11T12:34:43.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.623+0000 7f1abeffd700 1 -- 172.21.15.47:0/180483712 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac400be40 con 0x7f1ac806c030 2024-05-11T12:34:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.623+0000 7f1abeffd700 1 -- 172.21.15.47:0/180483712 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40206c0 con 0x7f1ac806c030 2024-05-11T12:34:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.624+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac810f1d0 con 0x7f1ac806c030 2024-05-11T12:34:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.627+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac810f660 con 0x7f1ac806c030 2024-05-11T12:34:43.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.639+0000 7f1abeffd700 1 -- 172.21.15.47:0/180483712 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ac4017a60 con 0x7f1ac806c030 2024-05-11T12:34:43.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.641+0000 7f1abeffd700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ab40b2a90 0x7f1ab40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.641+0000 7f1abeffd700 1 -- 172.21.15.47:0/180483712 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1ac40e35b0 con 0x7f1ac806c030 2024-05-11T12:34:43.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.642+0000 7f1ada7d1b80 1 Processor -- start 2024-05-11T12:34:43.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.642+0000 7f1ada7d1b80 1 -- start start 2024-05-11T12:34:43.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ab40b2a90 0x7f1ab40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610c03ad0 0x559610c03eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 0x559610bfe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610bfea60 0x559610c0a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559610b50860 con 0x559610c03ad0 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559610aef7c0 con 0x559610bfea60 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559610b4c820 con 0x559610bfe0a0 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1acdcd7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610c03ad0 0x559610c03eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1acdcd7700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610c03ad0 0x559610c03eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57772/0 (socket says 172.21.15.47:57772) 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.643+0000 7f1acdcd7700 1 -- 172.21.15.47:0/1471624175 learned_addr learned my addr 172.21.15.47:0/1471624175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.644+0000 7f1acdcd7700 1 -- 172.21.15.47:0/1471624175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 msgr2=0x559610bfe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.644+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/1471624175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 0x559610bfe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.644+0000 7f1acdcd7700 1 -- 172.21.15.47:0/1471624175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610bfea60 msgr2=0x559610c0a890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:34:43.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.644+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/1471624175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610bfea60 0x559610c0a890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.644+0000 7f1acdcd7700 1 -- 172.21.15.47:0/1471624175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559610a9d8d0 con 0x559610c03ad0 2024-05-11T12:34:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.644+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ab40b2a90 0x7f1ab40b4f50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1ac806bec0 tx=0x7f1ab80091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.644+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/1471624175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610c03ad0 0x559610c03eb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40aa830 tx=0x7f1ac4005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4945e169f08db866 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:43.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.646+0000 7f1ab3fff700 1 -- 172.21.15.47:0/1471624175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40abe60 con 0x559610c03ad0 2024-05-11T12:34:43.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.646+0000 7f1ab3fff700 1 -- 172.21.15.47:0/1471624175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac40aa640 con 0x559610c03ad0 2024-05-11T12:34:43.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.646+0000 7f1ab3fff700 1 -- 172.21.15.47:0/1471624175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40b3900 con 0x559610c03ad0 2024-05-11T12:34:43.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/1471624175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610c03ad0 msgr2=0x559610c03eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/1471624175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610c03ad0 0x559610c03eb0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40aa830 tx=0x7f1ac4005d20 comp rx=0 tx=0).stop 2024-05-11T12:34:43.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/1471624175 shutdown_connections 2024-05-11T12:34:43.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/1471624175 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610bfea60 0x559610c0a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/1471624175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610c03ad0 0x559610c03eb0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/1471624175 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 0x559610bfe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/1471624175 >> 172.21.15.47:0/1471624175 conn(0x559610b2ae60 msgr2=0x559610bf8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/1471624175 shutdown_connections 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.648+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/1471624175 wait complete. 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.649+0000 7f1ada7d1b80 1 -- start start 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.649+0000 7f1ada7d1b80 1 Processor -- start 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 0x559610c0a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfea60 0x559610c07940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x559610c045e0 con 0x559610bfea60 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x559610ab6c30 con 0x559610c03ad0 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x559610c0add0 con 0x559610bfe0a0 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1ada7d1b80 10 client.0 _ll_register_callbacks cb 0x559610ae8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:43.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.650+0000 7f1acd4d6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfea60 0x559610c07940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1ace4d8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acd4d6700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfea60 0x559610c07940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57788/0 (socket says 172.21.15.47:57788) 2024-05-11T12:34:43.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acd4d6700 1 -- 172.21.15.47:0/263814834 learned_addr learned my addr 172.21.15.47:0/263814834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 0x559610c0a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acd4d6700 1 -- 172.21.15.47:0/263814834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 msgr2=0x559610c0a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 0x559610c0a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acd4d6700 1 -- 172.21.15.47:0/263814834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 msgr2=0x559610c07e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c07e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.651+0000 7f1acd4d6700 1 -- 172.21.15.47:0/263814834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559610aec690 con 0x559610bfea60 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.652+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfea60 0x559610c07940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8016ca0 tx=0x7f1ab8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c335eaff04dcd74 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:43.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.652+0000 7f1ab17fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:34:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.653+0000 7f1ab17fa700 1 -- 172.21.15.47:0/263814834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8025060 con 0x559610bfea60 2024-05-11T12:34:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.653+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac40aa640 con 0x559610bfea60 2024-05-11T12:34:43.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.653+0000 7f1ab17fa700 1 -- 172.21.15.47:0/263814834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1ab8018c60 con 0x559610bfea60 2024-05-11T12:34:43.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.654+0000 7f1ab17fa700 1 -- 172.21.15.47:0/263814834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1ab800f040 con 0x559610bfea60 2024-05-11T12:34:43.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.654+0000 7f1ada7d1b80 10 client.15891 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:43.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.654+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559610c154f0 con 0x559610bfea60 2024-05-11T12:34:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.655+0000 7f1ab17fa700 1 -- 172.21.15.47:0/263814834 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac40aa640 con 0x559610bfea60 2024-05-11T12:34:43.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.655+0000 7f1ab17fa700 1 -- 172.21.15.47:0/263814834 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1ab8012420 con 0x559610bfea60 2024-05-11T12:34:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.657+0000 7f1ada7d1b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.657+0000 7f1ada7d1b80 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.657+0000 7f1ada7d1b80 20 client.15891 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.657+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c17ed0 0x559610c1a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.657+0000 7f1ada7d1b80 4 client.15891 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:34:43.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.657+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559610b5e240 con 0x559610c17ed0 2024-05-11T12:34:43.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.669+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c17ed0 0x559610c1a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.681+0000 7f1ab17fa700 10 client.15891 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:43.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.681+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c17ed0 0x559610c1a2f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b42327660f5e96d server_cookie=e4db63026b186176 in_seq=0 out_seq=0 2024-05-11T12:34:43.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.684+0000 7f1ab17fa700 1 -- 172.21.15.47:0/263814834 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559610b5e240 con 0x559610c17ed0 2024-05-11T12:34:43.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.684+0000 7f1ab17fa700 10 client.15891 handle_command_reply: tid=0 2024-05-11T12:34:43.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.685+0000 7f1ada7d1b80 1 client.15891 shutdown 2024-05-11T12:34:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.688+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfea60 msgr2=0x559610c07940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.688+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfea60 0x559610c07940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8016ca0 tx=0x7f1ab8004770 comp rx=0 tx=0).stop 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 shutdown_connections 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c17ed0 0x559610c1a2f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c07e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfe0a0 0x559610c0a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/263814834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfea60 0x559610c07940 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 >> 172.21.15.47:0/263814834 conn(0x559610b2ae60 msgr2=0x559610bf3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ab17fa700 0 client.15891 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 shutdown_connections 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.691+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/263814834 wait complete. 2024-05-11T12:34:43.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.692+0000 7f1ada7d1b80 20 client.15891 trim_cache size 0 max 16384 2024-05-11T12:34:43.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.759+0000 7f1ada7d1b80 1 Processor -- start 2024-05-11T12:34:43.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.761+0000 7f1ada7d1b80 1 -- start start 2024-05-11T12:34:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.761+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 0x559610c05560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.761+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c13a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.761+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610bf2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.761+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81a1570 con 0x559610bfe0a0 2024-05-11T12:34:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.761+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81a3df0 con 0x559610c03ad0 2024-05-11T12:34:43.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.761+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81a1be0 con 0x559610bfea60 2024-05-11T12:34:43.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.763+0000 7f1acd4d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c13a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.763+0000 7f1acd4d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c13a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:43992/0 (socket says 172.21.15.47:43992) 2024-05-11T12:34:43.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.763+0000 7f1acd4d6700 1 -- 172.21.15.47:0/3959855293 learned_addr learned my addr 172.21.15.47:0/3959855293 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:43.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.763+0000 7f1ace4d8700 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610bf2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.764+0000 7f1ace4d8700 1 -- 172.21.15.47:0/3959855293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 msgr2=0x559610c13a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.764+0000 7f1ace4d8700 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c13a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.764+0000 7f1ace4d8700 1 -- 172.21.15.47:0/3959855293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 msgr2=0x559610c05560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.764+0000 7f1ace4d8700 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 0x559610c05560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.764+0000 7f1ace4d8700 1 -- 172.21.15.47:0/3959855293 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559610bf5590 con 0x559610c03ad0 2024-05-11T12:34:43.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.773+0000 7f1ace4d8700 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610bf2bd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0003e60 tx=0x7f1ac001ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de832d81724f36b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:43.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.779+0000 7f1ab37fe700 1 -- 172.21.15.47:0/3959855293 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004bb0 con 0x559610c03ad0 2024-05-11T12:34:43.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.779+0000 7f1ab37fe700 1 -- 172.21.15.47:0/3959855293 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac0004d50 con 0x559610c03ad0 2024-05-11T12:34:43.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.779+0000 7f1ab37fe700 1 -- 172.21.15.47:0/3959855293 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004730 con 0x559610c03ad0 2024-05-11T12:34:43.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.784+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/3959855293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 msgr2=0x559610bf2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.784+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610bf2bd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0003e60 tx=0x7f1ac001ab90 comp rx=0 tx=0).stop 2024-05-11T12:34:43.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.784+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/3959855293 shutdown_connections 2024-05-11T12:34:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.784+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610bf2bd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.784+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 0x559610c05560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.784+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/3959855293 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c13a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.784+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/3959855293 >> 172.21.15.47:0/3959855293 conn(0x559610b2ae60 msgr2=0x559610bfd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.787+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/3959855293 shutdown_connections 2024-05-11T12:34:43.794 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:43 smithi047 ceph-mon[30375]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 2.22k op/s 2024-05-11T12:34:43.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.791+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/3959855293 wait complete. 2024-05-11T12:34:43.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.792+0000 7f1ada7d1b80 1 -- start start 2024-05-11T12:34:43.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.792+0000 7f1ada7d1b80 1 Processor -- start 2024-05-11T12:34:43.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.793+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 0x559610c10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.793+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c1f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.793+0000 7f1ada7d1b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c1f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.793+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81a22b0 con 0x559610bfe0a0 2024-05-11T12:34:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.793+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81a3720 con 0x559610c03ad0 2024-05-11T12:34:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.793+0000 7f1ada7d1b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81a2980 con 0x559610bfea60 2024-05-11T12:34:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.793+0000 7f1ada7d1b80 10 client.0 _ll_register_callbacks cb 0x559610ae8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:34:43.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.794+0000 7f1acd4d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c1f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.794+0000 7f1acd4d6700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c1f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:43994/0 (socket says 172.21.15.47:43994) 2024-05-11T12:34:43.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.794+0000 7f1acd4d6700 1 -- 172.21.15.47:0/909003097 learned_addr learned my addr 172.21.15.47:0/909003097 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.796+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 0x559610c10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.796+0000 7f1ace4d8700 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c1f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.796+0000 7f1acd4d6700 1 -- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 msgr2=0x559610c1f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.796+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c1f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.796+0000 7f1acd4d6700 1 -- 172.21.15.47:0/909003097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 msgr2=0x559610c10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:34:43.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.796+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 0x559610c10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.796+0000 7f1acd4d6700 1 -- 172.21.15.47:0/909003097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559610c817b0 con 0x559610bfea60 2024-05-11T12:34:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.799+0000 7f1acd4d6700 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c1f210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8012960 tx=0x7f1ab80126a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f85c73af1a126d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:43.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.799+0000 7f1ab17fa700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:34:43.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.800+0000 7f1ab17fa700 1 -- 172.21.15.47:0/909003097 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8009600 con 0x559610bfea60 2024-05-11T12:34:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.800+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac0004d50 con 0x559610bfea60 2024-05-11T12:34:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.801+0000 7f1ab17fa700 1 -- 172.21.15.47:0/909003097 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1ab802a460 con 0x559610bfea60 2024-05-11T12:34:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.802+0000 7f1ab17fa700 1 -- 172.21.15.47:0/909003097 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1ab802a680 con 0x559610bfea60 2024-05-11T12:34:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.802+0000 7f1ada7d1b80 10 client.25483 fetch_fsmap learned FSMap version 20 2024-05-11T12:34:43.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.802+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559610c828b0 con 0x559610bfea60 2024-05-11T12:34:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.803+0000 7f1ab17fa700 1 -- 172.21.15.47:0/909003097 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac0004d50 con 0x559610bfea60 2024-05-11T12:34:43.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.804+0000 7f1ab17fa700 1 -- 172.21.15.47:0/909003097 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f1ab8033820 con 0x559610bfea60 2024-05-11T12:34:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.804+0000 7f1ada7d1b80 10 client.25483 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:34:43.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.805+0000 7f1ada7d1b80 10 client.25483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:34:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.805+0000 7f1ada7d1b80 20 client.25483 populate_metadata read hostname 'smithi047' 2024-05-11T12:34:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.805+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c82ea0 0x559610c852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.805+0000 7f1ada7d1b80 4 client.25483 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:34:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.805+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559610b582e0 con 0x559610c82ea0 2024-05-11T12:34:43.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.810+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c82ea0 0x559610c852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.826+0000 7f1ab17fa700 10 client.25483 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:43.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.826+0000 7f1acdcd7700 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c82ea0 0x559610c852c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9214a2e6c97f79e server_cookie=2bc96c53baa8bad9 in_seq=0 out_seq=0 2024-05-11T12:34:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.830+0000 7f1ab17fa700 1 -- 172.21.15.47:0/909003097 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559610b582e0 con 0x559610c82ea0 2024-05-11T12:34:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.830+0000 7f1ab17fa700 10 client.25483 handle_command_reply: tid=0 2024-05-11T12:34:43.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.830+0000 7f1ada7d1b80 1 client.25483 shutdown 2024-05-11T12:34:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 msgr2=0x559610c1f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c1f210 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8012960 tx=0x7f1ab80126a0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 shutdown_connections 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x559610c82ea0 0x559610c852c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x559610c03ad0 0x559610c1f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x559610bfe0a0 0x559610c10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 --2- 172.21.15.47:0/909003097 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x559610bfea60 0x559610c1f210 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.831+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 >> 172.21.15.47:0/909003097 conn(0x559610b2ae60 msgr2=0x559610bf2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.832+0000 7f1ab17fa700 0 client.25483 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.832+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 shutdown_connections 2024-05-11T12:34:43.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.832+0000 7f1ada7d1b80 1 -- 172.21.15.47:0/909003097 wait complete. 2024-05-11T12:34:43.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.839+0000 7f1ada7d1b80 20 client.25483 trim_cache size 0 max 16384 2024-05-11T12:34:43.842 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:34:43.842 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:34:43.842 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:34:43.842 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:34:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ab40b2a90 msgr2=0x7f1ab40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ab40b2a90 0x7f1ab40b4f50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1ac806bec0 tx=0x7f1ab80091f0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 msgr2=0x7f1ac8074010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 0x7f1ac8074010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000c00 tx=0x7f1ac400b840 comp rx=0 tx=0).stop 2024-05-11T12:34:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 shutdown_connections 2024-05-11T12:34:43.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1ac8108470 0x7f1ac810ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1ab40b2a90 0x7f1ab40b4f50 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1ac806bec0 tx=0x7f1ab80091f0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ac806c030 0x7f1ac8074010 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000c00 tx=0x7f1ac400b840 comp rx=0 tx=0).stop 2024-05-11T12:34:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 --2- 172.21.15.47:0/180483712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1ac8105f80 0x7f1ac8074550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.845+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 >> 172.21.15.47:0/180483712 conn(0x7f1ac80793c0 msgr2=0x7f1ac8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:43.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.846+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 shutdown_connections 2024-05-11T12:34:43.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:43.850+0000 7f1acff3b700 1 -- 172.21.15.47:0/180483712 wait complete. 2024-05-11T12:34:43.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:43 smithi050 ceph-mon[34072]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 2.22k op/s 2024-05-11T12:34:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:43 smithi064 ceph-mon[34537]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 850 MiB/s wr, 2.22k op/s 2024-05-11T12:34:44.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:44 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:34:45.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:44 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:34:45.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:44 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:34:45.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:45 smithi050 ceph-mon[34072]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 715 MiB/s wr, 1.77k op/s 2024-05-11T12:34:46.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:45 smithi047 ceph-mon[30375]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 715 MiB/s wr, 1.77k op/s 2024-05-11T12:34:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:45 smithi064 ceph-mon[34537]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 715 MiB/s wr, 1.77k op/s 2024-05-11T12:34:46.217 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:34:46.217 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:34:47.072 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:34:48.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:47 smithi047 ceph-mon[30375]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 860 MiB/s wr, 1.88k op/s 2024-05-11T12:34:48.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:47 smithi064 ceph-mon[34537]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 860 MiB/s wr, 1.88k op/s 2024-05-11T12:34:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:47 smithi050 ceph-mon[34072]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 860 MiB/s wr, 1.88k op/s 2024-05-11T12:34:49.829 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:49 smithi047 ceph-mon[30375]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.94k op/s 2024-05-11T12:34:50.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:49 smithi064 ceph-mon[34537]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.94k op/s 2024-05-11T12:34:50.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:49 smithi050 ceph-mon[34072]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 819 MiB/s wr, 1.94k op/s 2024-05-11T12:34:52.064 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:51 smithi047 ceph-mon[30375]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 721 MiB/s wr, 1.91k op/s 2024-05-11T12:34:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:51 smithi064 ceph-mon[34537]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 721 MiB/s wr, 1.91k op/s 2024-05-11T12:34:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:51 smithi050 ceph-mon[34072]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 721 MiB/s wr, 1.91k op/s 2024-05-11T12:34:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.805+0000 7f876d3e1700 1 -- 172.21.15.47:0/3177917149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810a790 msgr2=0x7f876810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.805+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3177917149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810a790 0x7f876810e7e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f875c0099c0 tx=0x7f875c009cd0 comp rx=0 tx=0).stop 2024-05-11T12:34:52.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.806+0000 7f876d3e1700 1 -- 172.21.15.47:0/3177917149 shutdown_connections 2024-05-11T12:34:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.806+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3177917149 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f876810eec0 0x7f8768116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.806+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3177917149 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8768109de0 0x7f876810a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.806+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3177917149 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810a790 0x7f876810e7e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:52.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.806+0000 7f876d3e1700 1 -- 172.21.15.47:0/3177917149 >> 172.21.15.47:0/3177917149 conn(0x7f8768076d20 msgr2=0x7f8768079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:52.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.810+0000 7f876d3e1700 1 -- 172.21.15.47:0/3177917149 shutdown_connections 2024-05-11T12:34:52.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.815+0000 7f876d3e1700 1 -- 172.21.15.47:0/3177917149 wait complete. 2024-05-11T12:34:52.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.816+0000 7f876d3e1700 1 Processor -- start 2024-05-11T12:34:52.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.825+0000 7f876d3e1700 1 -- start start 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.826+0000 7f876d3e1700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8768109de0 0x7f87681a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.826+0000 7f876d3e1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f876810a790 0x7f87681a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.826+0000 7f876d3e1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810eec0 0x7f87681a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.826+0000 7f876d3e1700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8768119dd0 con 0x7f876810eec0 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.826+0000 7f876d3e1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f8768119c50 con 0x7f876810a790 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.826+0000 7f876d3e1700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f8768119f50 con 0x7f8768109de0 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.827+0000 7f87677fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810eec0 0x7f87681a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.827+0000 7f87677fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810eec0 0x7f87681a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60416/0 (socket says 172.21.15.47:60416) 2024-05-11T12:34:52.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.827+0000 7f87677fe700 1 -- 172.21.15.47:0/3700572161 learned_addr learned my addr 172.21.15.47:0/3700572161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:34:52.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.830+0000 7f87677fe700 1 -- 172.21.15.47:0/3700572161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8768109de0 msgr2=0x7f87681a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:52.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.830+0000 7f87677fe700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8768109de0 0x7f87681a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:52.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.830+0000 7f87677fe700 1 -- 172.21.15.47:0/3700572161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f876810a790 msgr2=0x7f87681a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:52.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.830+0000 7f87677fe700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f876810a790 0x7f87681a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:52.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.830+0000 7f87677fe700 1 -- 172.21.15.47:0/3700572161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87681a52c0 con 0x7f876810eec0 2024-05-11T12:34:52.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.831+0000 7f87677fe700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810eec0 0x7f87681a4b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f876000ada0 tx=0x7f876000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:52.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.833+0000 7f874ffff700 1 -- 172.21.15.47:0/3700572161 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760017020 con 0x7f876810eec0 2024-05-11T12:34:52.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.833+0000 7f874ffff700 1 -- 172.21.15.47:0/3700572161 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f876000ce30 con 0x7f876810eec0 2024-05-11T12:34:52.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.833+0000 7f874ffff700 1 -- 172.21.15.47:0/3700572161 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760005520 con 0x7f876810eec0 2024-05-11T12:34:52.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.833+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87681a55b0 con 0x7f876810eec0 2024-05-11T12:34:52.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.833+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87681ace30 con 0x7f876810eec0 2024-05-11T12:34:52.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.840+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8754005330 con 0x7f876810eec0 2024-05-11T12:34:52.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.867+0000 7f874ffff700 1 -- 172.21.15.47:0/3700572161 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f876000c980 con 0x7f876810eec0 2024-05-11T12:34:52.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.869+0000 7f874ffff700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87500b2a90 0x7f87500b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:34:52.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.870+0000 7f874ffff700 1 -- 172.21.15.47:0/3700572161 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f87600e1e10 con 0x7f876810eec0 2024-05-11T12:34:52.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.871+0000 7f874ffff700 1 -- 172.21.15.47:0/3700572161 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8760021020 con 0x7f876810eec0 2024-05-11T12:34:52.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.872+0000 7f8766ffd700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87500b2a90 0x7f87500b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:34:52.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:52.880+0000 7f8766ffd700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87500b2a90 0x7f87500b4f50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f875800a990 tx=0x7f8758005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:34:53.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.477+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8754005620 con 0x7f876810eec0 2024-05-11T12:34:53.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.484+0000 7f874ffff700 1 -- 172.21.15.47:0/3700572161 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f87600a9b10 con 0x7f876810eec0 2024-05-11T12:34:53.489 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:34:53.489 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:34:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.491+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87500b2a90 msgr2=0x7f87500b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.491+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87500b2a90 0x7f87500b4f50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f875800a990 tx=0x7f8758005c80 comp rx=0 tx=0).stop 2024-05-11T12:34:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.491+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810eec0 msgr2=0x7f87681a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:34:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.491+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810eec0 0x7f87681a4b80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f876000ada0 tx=0x7f876000c330 comp rx=0 tx=0).stop 2024-05-11T12:34:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.492+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 shutdown_connections 2024-05-11T12:34:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.492+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f876810eec0 0x7f87681a4b80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.492+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f8768109de0 0x7f87681a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.492+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f87500b2a90 0x7f87500b4f50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.492+0000 7f876d3e1700 1 --2- 172.21.15.47:0/3700572161 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f876810a790 0x7f87681a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:34:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.492+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 >> 172.21.15.47:0/3700572161 conn(0x7f8768076d20 msgr2=0x7f8768078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:34:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.497+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 shutdown_connections 2024-05-11T12:34:53.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:34:53.501+0000 7f876d3e1700 1 -- 172.21.15.47:0/3700572161 wait complete. 2024-05-11T12:34:53.505 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:34:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:53 smithi064 ceph-mon[34537]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 927 MiB/s wr, 2.57k op/s 2024-05-11T12:34:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:53 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3700572161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:34:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:53 smithi050 ceph-mon[34072]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 927 MiB/s wr, 2.57k op/s 2024-05-11T12:34:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:53 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3700572161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:34:54.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:53 smithi047 ceph-mon[30375]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 927 MiB/s wr, 2.57k op/s 2024-05-11T12:34:54.278 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:53 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3700572161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:34:55.561 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:34:56.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:55 smithi047 ceph-mon[30375]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 712 MiB/s wr, 1.74k op/s 2024-05-11T12:34:56.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:55 smithi064 ceph-mon[34537]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 712 MiB/s wr, 1.74k op/s 2024-05-11T12:34:56.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:55 smithi050 ceph-mon[34072]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 712 MiB/s wr, 1.74k op/s 2024-05-11T12:34:56.446 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:34:58.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:57 smithi064 ceph-mon[34537]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 2.05k op/s 2024-05-11T12:34:58.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:57 smithi050 ceph-mon[34072]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 2.05k op/s 2024-05-11T12:34:58.237 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:57 smithi047 ceph-mon[30375]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 797 MiB/s wr, 2.05k op/s 2024-05-11T12:35:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:34:59 smithi050 ceph-mon[34072]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 759 MiB/s wr, 2.57k op/s 2024-05-11T12:35:00.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:34:59 smithi047 ceph-mon[30375]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 759 MiB/s wr, 2.57k op/s 2024-05-11T12:35:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:34:59 smithi064 ceph-mon[34537]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 759 MiB/s wr, 2.57k op/s 2024-05-11T12:35:02.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.060+0000 7fc2729ad700 1 -- 172.21.15.47:0/2283643888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 msgr2=0x7fc26c10af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.060+0000 7fc2729ad700 1 --2- 172.21.15.47:0/2283643888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 0x7fc26c10af10 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc260009990 tx=0x7fc260009ca0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.063 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:01 smithi047 ceph-mon[30375]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 664 MiB/s wr, 2.47k op/s 2024-05-11T12:35:02.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.062+0000 7fc2729ad700 1 -- 172.21.15.47:0/2283643888 shutdown_connections 2024-05-11T12:35:02.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.062+0000 7fc2729ad700 1 --2- 172.21.15.47:0/2283643888 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c11cd30 0x7fc26c11f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.062+0000 7fc2729ad700 1 --2- 172.21.15.47:0/2283643888 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c111d30 0x7fc26c10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.062+0000 7fc2729ad700 1 --2- 172.21.15.47:0/2283643888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 0x7fc26c10af10 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.062+0000 7fc2729ad700 1 -- 172.21.15.47:0/2283643888 >> 172.21.15.47:0/2283643888 conn(0x7fc26c06e6c0 msgr2=0x7fc26c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:02.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.066+0000 7fc2729ad700 1 -- 172.21.15.47:0/2283643888 shutdown_connections 2024-05-11T12:35:02.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.066+0000 7fc2729ad700 1 -- 172.21.15.47:0/2283643888 wait complete. 2024-05-11T12:35:02.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.067+0000 7fc2729ad700 1 Processor -- start 2024-05-11T12:35:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.067+0000 7fc2729ad700 1 -- start start 2024-05-11T12:35:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc2729ad700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 0x7fc26c1ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc2729ad700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c111d30 0x7fc26c1aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc2729ad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c11cd30 0x7fc26c1a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc2729ad700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c122610 con 0x7fc26c10aa90 2024-05-11T12:35:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc2729ad700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c122490 con 0x7fc26c111d30 2024-05-11T12:35:02.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc2729ad700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c122790 con 0x7fc26c11cd30 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc26bfff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 0x7fc26c1ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c111d30 0x7fc26c1aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c111d30 0x7fc26c1aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:37462/0 (socket says 172.21.15.47:37462) 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.068+0000 7fc26b7fe700 1 -- 172.21.15.47:0/557804006 learned_addr learned my addr 172.21.15.47:0/557804006 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.069+0000 7fc26b7fe700 1 -- 172.21.15.47:0/557804006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c11cd30 msgr2=0x7fc26c1a88b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.069+0000 7fc26b7fe700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c11cd30 0x7fc26c1a88b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.070+0000 7fc26b7fe700 1 -- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 msgr2=0x7fc26c1ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.070+0000 7fc26b7fe700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 0x7fc26c1ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.070+0000 7fc26b7fe700 1 -- 172.21.15.47:0/557804006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c1a9010 con 0x7fc26c111d30 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.071+0000 7fc26b7fe700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c111d30 0x7fc26c1aec30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b2e0 tx=0x7fc2600057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.075+0000 7fc2697fa700 1 -- 172.21.15.47:0/557804006 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260026070 con 0x7fc26c111d30 2024-05-11T12:35:02.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.075+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c1a92a0 con 0x7fc26c111d30 2024-05-11T12:35:02.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.075+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c112510 con 0x7fc26c111d30 2024-05-11T12:35:02.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.075+0000 7fc2697fa700 1 -- 172.21.15.47:0/557804006 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc260003d30 con 0x7fc26c111d30 2024-05-11T12:35:02.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.075+0000 7fc2697fa700 1 -- 172.21.15.47:0/557804006 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260004230 con 0x7fc26c111d30 2024-05-11T12:35:02.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.086+0000 7fc27d243b80 1 Processor -- start 2024-05-11T12:35:02.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.095+0000 7fc27d243b80 1 -- start start 2024-05-11T12:35:02.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.095+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.095+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0f9eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.095+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9eba60 0x564b0f9f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.095+0000 7fc27d243b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564b0f93d860 con 0x564b0f9eba60 2024-05-11T12:35:02.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.095+0000 7fc27d243b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564b0f8dc7c0 con 0x564b0f9eb0a0 2024-05-11T12:35:02.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.095+0000 7fc27d243b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564b0f939820 con 0x564b0f9f0ad0 2024-05-11T12:35:02.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.096+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9eba60 0x564b0f9f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.099+0000 7fc270f4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.099+0000 7fc270f4a700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52570/0 (socket says 172.21.15.47:52570) 2024-05-11T12:35:02.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.099+0000 7fc270f4a700 1 -- 172.21.15.47:0/2570170861 learned_addr learned my addr 172.21.15.47:0/2570170861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:02.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.099+0000 7fc26bfff700 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0f9eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.100+0000 7fc2697fa700 1 -- 172.21.15.47:0/557804006 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc260009d10 con 0x7fc26c111d30 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.100+0000 7fc270f4a700 1 -- 172.21.15.47:0/2570170861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 msgr2=0x564b0f9eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.100+0000 7fc270f4a700 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0f9eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.100+0000 7fc270f4a700 1 -- 172.21.15.47:0/2570170861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9eba60 msgr2=0x564b0f9f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.100+0000 7fc270f4a700 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9eba60 0x564b0f9f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.100+0000 7fc270f4a700 1 -- 172.21.15.47:0/2570170861 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0f88a8d0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.102+0000 7fc2697fa700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2540b2a50 0x7fc2540b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.102+0000 7fc2697fa700 1 -- 172.21.15.47:0/557804006 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc2600e61f0 con 0x7fc26c111d30 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.102+0000 7fc270f4a700 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f0eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e3e0 tx=0x7fc26400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5872b9324629f7a server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:02.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.103+0000 7fc26bfff700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2540b2a50 0x7fc2540b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.104+0000 7fc26bfff700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2540b2a50 0x7fc2540b4f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc26c1a99f0 tx=0x7fc25c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.106+0000 7fc2527fc700 1 -- 172.21.15.47:0/2570170861 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26400ee30 con 0x564b0f9f0ad0 2024-05-11T12:35:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.107+0000 7fc2527fc700 1 -- 172.21.15.47:0/2570170861 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc264004bb0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.107+0000 7fc2527fc700 1 -- 172.21.15.47:0/2570170861 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2640042e0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.109+0000 7fc27d243b80 1 -- 172.21.15.47:0/2570170861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9f0ad0 msgr2=0x564b0f9f0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.109+0000 7fc27d243b80 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f0eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc26400e3e0 tx=0x7fc26400b040 comp rx=0 tx=0).stop 2024-05-11T12:35:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.128+0000 7fc27d243b80 1 -- 172.21.15.47:0/2570170861 shutdown_connections 2024-05-11T12:35:02.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.128+0000 7fc27d243b80 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9eba60 0x564b0f9f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.128+0000 7fc27d243b80 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f0eb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.128+0000 7fc27d243b80 1 --2- 172.21.15.47:0/2570170861 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0f9eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.128+0000 7fc27d243b80 1 -- 172.21.15.47:0/2570170861 >> 172.21.15.47:0/2570170861 conn(0x564b0f917e60 msgr2=0x564b0f9e5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:02.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.133+0000 7fc27d243b80 1 -- 172.21.15.47:0/2570170861 shutdown_connections 2024-05-11T12:35:02.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.134+0000 7fc27d243b80 1 -- 172.21.15.47:0/2570170861 wait complete. 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.134+0000 7fc27d243b80 1 -- start start 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.135+0000 7fc27d243b80 1 Processor -- start 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.136+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.136+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.136+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.136+0000 7fc27d243b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x564b0f9f7e30 con 0x564b0f9f0ad0 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.136+0000 7fc27d243b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x564b0f8a3c30 con 0x564b0f9eba60 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.136+0000 7fc27d243b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x564b0f9f7fb0 con 0x564b0f9eb0a0 2024-05-11T12:35:02.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.136+0000 7fc27d243b80 10 client.0 _ll_register_callbacks cb 0x564b0f8d5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:02.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.138+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.139+0000 7fc270f4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.139+0000 7fc270f4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:36638/0 (socket says 172.21.15.47:36638) 2024-05-11T12:35:02.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.139+0000 7fc270f4a700 1 -- 172.21.15.47:0/500264271 learned_addr learned my addr 172.21.15.47:0/500264271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:02.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.143+0000 7fc26bfff700 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.144+0000 7fc270f4a700 1 -- 172.21.15.47:0/500264271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 msgr2=0x564b0f9f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.144+0000 7fc270f4a700 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.144+0000 7fc270f4a700 1 -- 172.21.15.47:0/500264271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 msgr2=0x564b0f9f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.144+0000 7fc270f4a700 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.144+0000 7fc270f4a700 1 -- 172.21.15.47:0/500264271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0f8d9690 con 0x564b0f9f0ad0 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.144+0000 7fc270f4a700 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f4d80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc264009110 tx=0x7fc264003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60fa3a6e1472f56 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.144+0000 7fc24b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.145+0000 7fc24b7fe700 1 -- 172.21.15.47:0/500264271 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264022030 con 0x564b0f9f0ad0 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.145+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc264004bb0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.145+0000 7fc24b7fe700 1 -- 172.21.15.47:0/500264271 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc264019070 con 0x564b0f9f0ad0 2024-05-11T12:35:02.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.146+0000 7fc24b7fe700 1 -- 172.21.15.47:0/500264271 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc264004d60 con 0x564b0f9f0ad0 2024-05-11T12:35:02.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.147+0000 7fc27d243b80 10 client.15912 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:02.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.148+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564b0fa02690 con 0x564b0f9f0ad0 2024-05-11T12:35:02.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.150+0000 7fc24b7fe700 1 -- 172.21.15.47:0/500264271 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc264004bb0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.151+0000 7fc24b7fe700 1 -- 172.21.15.47:0/500264271 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc264007720 con 0x564b0f9f0ad0 2024-05-11T12:35:02.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.151+0000 7fc27d243b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:02.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.151+0000 7fc27d243b80 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:02.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.151+0000 7fc27d243b80 20 client.15912 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:02.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.151+0000 7fc27d243b80 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa05000 0x564b0fa07460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.151+0000 7fc27d243b80 4 client.15912 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:35:02.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.151+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b0f94b240 con 0x564b0fa05000 2024-05-11T12:35:02.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.154+0000 7fc26bfff700 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa05000 0x564b0fa07460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.160+0000 7fc26bfff700 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa05000 0x564b0fa07460 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caa4aa44d6c0fb2 server_cookie=fe39edd0217446ae in_seq=0 out_seq=0 2024-05-11T12:35:02.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.161+0000 7fc24b7fe700 10 client.15912 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:02.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.166+0000 7fc24b7fe700 1 -- 172.21.15.47:0/500264271 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b0f94b240 con 0x564b0fa05000 2024-05-11T12:35:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.166+0000 7fc24b7fe700 10 client.15912 handle_command_reply: tid=0 2024-05-11T12:35:02.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.166+0000 7fc27d243b80 1 client.15912 shutdown 2024-05-11T12:35:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.168+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 msgr2=0x564b0f9f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.168+0000 7fc27d243b80 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f4d80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc264009110 tx=0x7fc264003f90 comp rx=0 tx=0).stop 2024-05-11T12:35:02.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.174+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 shutdown_connections 2024-05-11T12:35:02.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.174+0000 7fc27d243b80 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa05000 0x564b0fa07460 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.174+0000 7fc27d243b80 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f4d80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.174+0000 7fc27d243b80 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.174+0000 7fc27d243b80 1 --2- 172.21.15.47:0/500264271 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.174+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 >> 172.21.15.47:0/500264271 conn(0x564b0f917e60 msgr2=0x564b0f9e0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:02.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.174+0000 7fc24b7fe700 0 client.15912 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:02.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.179+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 shutdown_connections 2024-05-11T12:35:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.183+0000 7fc27d243b80 1 -- 172.21.15.47:0/500264271 wait complete. 2024-05-11T12:35:02.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.184+0000 7fc27d243b80 20 client.15912 trim_cache size 0 max 16384 2024-05-11T12:35:02.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:01 smithi050 ceph-mon[34072]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 664 MiB/s wr, 2.47k op/s 2024-05-11T12:35:02.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.227+0000 7fc27d243b80 1 Processor -- start 2024-05-11T12:35:02.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.228+0000 7fc27d243b80 1 -- start start 2024-05-11T12:35:02.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.229+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0fa00f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.229+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eba60 0x564b0f9f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.230+0000 7fc26bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0fa00f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.230+0000 7fc26bfff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0fa00f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:37486/0 (socket says 172.21.15.47:37486) 2024-05-11T12:35:02.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.231+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eba60 0x564b0f9f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.231+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eba60 0x564b0f9f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52598/0 (socket says 172.21.15.47:52598) 2024-05-11T12:35:02.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.231+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.232+0000 7fc270f4a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.232+0000 7fc26bfff700 1 -- 172.21.15.47:0/1967123145 learned_addr learned my addr 172.21.15.47:0/1967123145 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.232+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc25800b710 con 0x564b0f9f0ad0 2024-05-11T12:35:02.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.233+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc25800dee0 con 0x564b0f9eb0a0 2024-05-11T12:35:02.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.233+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc25800bcd0 con 0x564b0f9eba60 2024-05-11T12:35:02.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.234+0000 7fc270f4a700 1 -- 172.21.15.47:0/1967123145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eba60 msgr2=0x564b0f9f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.234+0000 7fc270f4a700 1 --2- 172.21.15.47:0/1967123145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eba60 0x564b0f9f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.234+0000 7fc270f4a700 1 -- 172.21.15.47:0/1967123145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 msgr2=0x564b0fa00f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.234+0000 7fc270f4a700 1 --2- 172.21.15.47:0/1967123145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0fa00f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.234+0000 7fc270f4a700 1 -- 172.21.15.47:0/1967123145 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0f9e21b0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.236+0000 7fc270f4a700 1 --2- 172.21.15.47:0/1967123145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f02e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc264009960 tx=0x7fc264009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7272fb4debd74c30 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:02.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.237+0000 7fc251ffb700 1 -- 172.21.15.47:0/1967123145 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264022040 con 0x564b0f9f0ad0 2024-05-11T12:35:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.237+0000 7fc251ffb700 1 -- 172.21.15.47:0/1967123145 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc264007d60 con 0x564b0f9f0ad0 2024-05-11T12:35:02.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.237+0000 7fc251ffb700 1 -- 172.21.15.47:0/1967123145 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26401e8d0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.239+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 msgr2=0x564b0f9f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.240+0000 7fc27d243b80 1 --2- 172.21.15.47:0/1967123145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f02e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc264009960 tx=0x7fc264009f20 comp rx=0 tx=0).stop 2024-05-11T12:35:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.240+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 shutdown_connections 2024-05-11T12:35:02.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.241+0000 7fc27d243b80 1 --2- 172.21.15.47:0/1967123145 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f02e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.241+0000 7fc27d243b80 1 --2- 172.21.15.47:0/1967123145 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eb0a0 0x564b0fa00f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.241+0000 7fc27d243b80 1 --2- 172.21.15.47:0/1967123145 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eba60 0x564b0f9f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.241+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 >> 172.21.15.47:0/1967123145 conn(0x564b0f917e60 msgr2=0x564b0f9ea090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.241+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 shutdown_connections 2024-05-11T12:35:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.242+0000 7fc27d243b80 1 -- 172.21.15.47:0/1967123145 wait complete. 2024-05-11T12:35:02.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.242+0000 7fc27d243b80 1 -- start start 2024-05-11T12:35:02.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.244+0000 7fc27d243b80 1 Processor -- start 2024-05-11T12:35:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.245+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.246+0000 7fc27d243b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.247+0000 7fc26bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.247+0000 7fc26bfff700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:52602/0 (socket says 172.21.15.47:52602) 2024-05-11T12:35:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.248+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.248+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:37494/0 (socket says 172.21.15.47:37494) 2024-05-11T12:35:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.248+0000 7fc26bfff700 1 -- 172.21.15.47:0/562692459 learned_addr learned my addr 172.21.15.47:0/562692459 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.249+0000 7fc27d243b80 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.249+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc25800c3a0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.250+0000 7fc270f4a700 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.251+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc25800d810 con 0x564b0f9eba60 2024-05-11T12:35:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.251+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc25800ca70 con 0x564b0f9eb0a0 2024-05-11T12:35:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.252+0000 7fc27d243b80 10 client.0 _ll_register_callbacks cb 0x564b0f8d5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.252+0000 7fc270f4a700 1 -- 172.21.15.47:0/562692459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 msgr2=0x564b0f9fd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.252+0000 7fc270f4a700 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9fd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.252+0000 7fc270f4a700 1 -- 172.21.15.47:0/562692459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 msgr2=0x564b0f9f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:02.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.252+0000 7fc270f4a700 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.252+0000 7fc270f4a700 1 -- 172.21.15.47:0/562692459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0f9f67e0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.253+0000 7fc270f4a700 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f6170 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x564b0f939970 tx=0x7fc26401e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09fb9d86a489682 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:02.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.253+0000 7fc24b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:02.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.254+0000 7fc24b7fe700 1 -- 172.21.15.47:0/562692459 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264022070 con 0x564b0f9f0ad0 2024-05-11T12:35:02.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.254+0000 7fc24b7fe700 1 -- 172.21.15.47:0/562692459 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc26401e670 con 0x564b0f9f0ad0 2024-05-11T12:35:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.254+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc264007d60 con 0x564b0f9f0ad0 2024-05-11T12:35:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.255+0000 7fc24b7fe700 1 -- 172.21.15.47:0/562692459 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc264004b90 con 0x564b0f9f0ad0 2024-05-11T12:35:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.255+0000 7fc27d243b80 10 client.15918 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.255+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({fsmap=20}) -- 0x564b0fa6e8e0 con 0x564b0f9f0ad0 2024-05-11T12:35:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.257+0000 7fc24b7fe700 1 -- 172.21.15.47:0/562692459 <== mon.0 v2:172.21.15.47:3300/0 4 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc26401e870 con 0x564b0f9f0ad0 2024-05-11T12:35:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.257+0000 7fc27d243b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.257+0000 7fc27d243b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.257+0000 7fc27d243b80 20 client.15918 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.257+0000 7fc27d243b80 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa6eb60 0x564b0fa70f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.257+0000 7fc27d243b80 4 client.15918 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:35:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.257+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc264007d60 con 0x564b0fa6eb60 2024-05-11T12:35:02.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.258+0000 7fc26bfff700 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa6eb60 0x564b0fa70f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:02.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.266+0000 7fc26bfff700 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa6eb60 0x564b0fa70f80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96898e31e7891669 server_cookie=2ef8a2e54f3b3a52 in_seq=0 out_seq=0 2024-05-11T12:35:02.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.266+0000 7fc24b7fe700 10 client.15918 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.271+0000 7fc24b7fe700 1 -- 172.21.15.47:0/562692459 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc264007d60 con 0x564b0fa6eb60 2024-05-11T12:35:02.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.271+0000 7fc24b7fe700 10 client.15918 handle_command_reply: tid=0 2024-05-11T12:35:02.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.272+0000 7fc27d243b80 1 client.15918 shutdown 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.272+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 msgr2=0x564b0f9f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f6170 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x564b0f939970 tx=0x7fc26401e4e0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 shutdown_connections 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x564b0fa6eb60 0x564b0fa70f80 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc24b7fe700 0 client.15918 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x564b0f9f0ad0 0x564b0f9f6170 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x564b0f9eb0a0 0x564b0f9fd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 --2- 172.21.15.47:0/562692459 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x564b0f9eba60 0x564b0f9f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 >> 172.21.15.47:0/562692459 conn(0x564b0f917e60 msgr2=0x564b0f9e0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.273+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 shutdown_connections 2024-05-11T12:35:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.274+0000 7fc27d243b80 1 -- 172.21.15.47:0/562692459 wait complete. 2024-05-11T12:35:02.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.274+0000 7fc27d243b80 20 client.15918 trim_cache size 0 max 16384 2024-05-11T12:35:02.281 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:35:02.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.284+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2540b2a50 msgr2=0x7fc2540b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.284+0000 7fc2729ad700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2540b2a50 0x7fc2540b4f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc26c1a99f0 tx=0x7fc25c009260 comp rx=0 tx=0).stop 2024-05-11T12:35:02.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.284+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c111d30 msgr2=0x7fc26c1aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:02.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.284+0000 7fc2729ad700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c111d30 0x7fc26c1aec30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b2e0 tx=0x7fc2600057c0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.286+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 shutdown_connections 2024-05-11T12:35:02.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.286+0000 7fc2729ad700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc26c11cd30 0x7fc26c1a88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.286+0000 7fc2729ad700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c10aa90 0x7fc26c1ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.286+0000 7fc2729ad700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc2540b2a50 0x7fc2540b4f10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.286+0000 7fc2729ad700 1 --2- 172.21.15.47:0/557804006 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc26c111d30 0x7fc26c1aec30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:02.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.286+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 >> 172.21.15.47:0/557804006 conn(0x7fc26c06e6c0 msgr2=0x7fc26c11ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.292+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 shutdown_connections 2024-05-11T12:35:02.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:02.292+0000 7fc2729ad700 1 -- 172.21.15.47:0/557804006 wait complete. 2024-05-11T12:35:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:01 smithi064 ceph-mon[34537]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 664 MiB/s wr, 2.47k op/s 2024-05-11T12:35:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:03 smithi050 ceph-mon[34072]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 2.87k op/s 2024-05-11T12:35:04.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:03 smithi047 ceph-mon[30375]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 2.87k op/s 2024-05-11T12:35:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:03 smithi064 ceph-mon[34537]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 882 MiB/s wr, 2.87k op/s 2024-05-11T12:35:05.973 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:35:05.974 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:35:06.166 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:05 smithi064 ceph-mon[34537]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 667 MiB/s wr, 2.11k op/s 2024-05-11T12:35:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:05 smithi050 ceph-mon[34072]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 667 MiB/s wr, 2.11k op/s 2024-05-11T12:35:06.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:05 smithi047 ceph-mon[30375]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 667 MiB/s wr, 2.11k op/s 2024-05-11T12:35:07.003 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:35:07.979 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:07 smithi047 ceph-mon[30375]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 768 MiB/s wr, 2.31k op/s 2024-05-11T12:35:08.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:07 smithi064 ceph-mon[34537]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 768 MiB/s wr, 2.31k op/s 2024-05-11T12:35:08.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:07 smithi050 ceph-mon[34072]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 768 MiB/s wr, 2.31k op/s 2024-05-11T12:35:10.279 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:10 smithi047 ceph-mon[30375]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 MiB/s wr, 2.30k op/s 2024-05-11T12:35:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:10 smithi064 ceph-mon[34537]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 MiB/s wr, 2.30k op/s 2024-05-11T12:35:10.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:10 smithi050 ceph-mon[34072]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 802 MiB/s wr, 2.30k op/s 2024-05-11T12:35:11.049 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:11 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:35:11.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:11 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:35:11.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:11 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:35:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.936+0000 7f4cb3c3e700 1 -- 172.21.15.47:0/3185566712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac11cd30 msgr2=0x7f4cac11f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:11.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.936+0000 7f4cb3c3e700 1 --2- 172.21.15.47:0/3185566712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac11cd30 0x7f4cac11f120 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0098f0 tx=0x7f4c9c009c00 comp rx=0 tx=0).stop 2024-05-11T12:35:11.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.940+0000 7f4cb3c3e700 1 -- 172.21.15.47:0/3185566712 shutdown_connections 2024-05-11T12:35:11.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.940+0000 7f4cb3c3e700 1 --2- 172.21.15.47:0/3185566712 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac11cd30 0x7f4cac11f120 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.940+0000 7f4cb3c3e700 1 --2- 172.21.15.47:0/3185566712 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cac111d30 0x7f4cac10a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.940+0000 7f4cb3c3e700 1 --2- 172.21.15.47:0/3185566712 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac10aa90 0x7f4cac10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.940+0000 7f4cb3c3e700 1 -- 172.21.15.47:0/3185566712 >> 172.21.15.47:0/3185566712 conn(0x7f4cac06e6c0 msgr2=0x7f4cac06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:11.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.942+0000 7f4cb3c3e700 1 -- 172.21.15.47:0/3185566712 shutdown_connections 2024-05-11T12:35:11.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.944+0000 7f4cb3c3e700 1 -- 172.21.15.47:0/3185566712 wait complete. 2024-05-11T12:35:11.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.946+0000 7f4cb3c3e700 1 Processor -- start 2024-05-11T12:35:11.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.947+0000 7f4cb3c3e700 1 -- start start 2024-05-11T12:35:11.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.949+0000 7f4cb3c3e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cac10aa90 0x7f4cac1ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:11.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.949+0000 7f4cb3c3e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac111d30 0x7f4cac1aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.949+0000 7f4cb3c3e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac11cd30 0x7f4cac1a8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.949+0000 7f4cb3c3e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac122660 con 0x7f4cac111d30 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.949+0000 7f4cb3c3e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac1224e0 con 0x7f4cac10aa90 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.950+0000 7f4cb3c3e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac1227e0 con 0x7f4cac11cd30 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.951+0000 7f4cb21db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac11cd30 0x7f4cac1a8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.951+0000 7f4cb21db700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac11cd30 0x7f4cac1a8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:45812/0 (socket says 172.21.15.47:45812) 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.952+0000 7f4cb21db700 1 -- 172.21.15.47:0/3605351937 learned_addr learned my addr 172.21.15.47:0/3605351937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:11.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.952+0000 7f4cb11d9700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac111d30 0x7f4cac1aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:11.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.954+0000 7f4cb19da700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cac10aa90 0x7f4cac1ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:11.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.957+0000 7f4cb21db700 1 -- 172.21.15.47:0/3605351937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cac10aa90 msgr2=0x7f4cac1ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:11.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.957+0000 7f4cb21db700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cac10aa90 0x7f4cac1ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.958+0000 7f4cb21db700 1 -- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac111d30 msgr2=0x7f4cac1aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:11.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.958+0000 7f4cb21db700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac111d30 0x7f4cac1aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.958+0000 7f4cb21db700 1 -- 172.21.15.47:0/3605351937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac1a90e0 con 0x7f4cac11cd30 2024-05-11T12:35:11.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.960+0000 7f4cb21db700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac11cd30 0x7f4cac1a8980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00b2e0 tx=0x7f4c9c00b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.962+0000 7f4ca2ffd700 1 -- 172.21.15.47:0/3605351937 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c027070 con 0x7f4cac11cd30 2024-05-11T12:35:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.962+0000 7f4ca2ffd700 1 -- 172.21.15.47:0/3605351937 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c9c022420 con 0x7f4cac11cd30 2024-05-11T12:35:11.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.962+0000 7f4ca2ffd700 1 -- 172.21.15.47:0/3605351937 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c00bea0 con 0x7f4cac11cd30 2024-05-11T12:35:11.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.966+0000 7f4cb3c3e700 1 -- 172.21.15.47:0/3605351937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac1a9370 con 0x7f4cac11cd30 2024-05-11T12:35:11.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.968+0000 7f4cb3c3e700 1 -- 172.21.15.47:0/3605351937 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1b5690 con 0x7f4cac11cd30 2024-05-11T12:35:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.977+0000 7f4cbe4d4b80 1 Processor -- start 2024-05-11T12:35:11.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.978+0000 7f4cbe4d4b80 1 -- start start 2024-05-11T12:35:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.986+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f112728eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.986+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f112723120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.986+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112723ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.986+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f1126758e0 con 0x55f1127293f0 2024-05-11T12:35:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.986+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f112614840 con 0x55f112723660 2024-05-11T12:35:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.986+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f1126718a0 con 0x55f112728ad0 2024-05-11T12:35:11.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.987+0000 7f4cb11d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f112728eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.988+0000 7f4cb11d9700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f112728eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:45816/0 (socket says 172.21.15.47:45816) 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.988+0000 7f4cb11d9700 1 -- 172.21.15.47:0/3490170135 learned_addr learned my addr 172.21.15.47:0/3490170135 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.988+0000 7f4cb11d9700 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112723ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.989+0000 7f4cb11d9700 1 -- 172.21.15.47:0/3490170135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 msgr2=0x55f112723ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.989+0000 7f4cb11d9700 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112723ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.989+0000 7f4cb11d9700 1 -- 172.21.15.47:0/3490170135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 msgr2=0x55f112723120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.989+0000 7f4cb11d9700 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f112723120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.989+0000 7f4cb11d9700 1 -- 172.21.15.47:0/3490170135 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1125c2950 con 0x55f112728ad0 2024-05-11T12:35:11.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.990+0000 7f4ca2ffd700 1 -- 172.21.15.47:0/3605351937 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f4c9c0225d0 con 0x7f4cac11cd30 2024-05-11T12:35:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.991+0000 7f4cb11d9700 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f112728eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400a800 tx=0x7f4ca40096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1003e05a9d1706d server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.991+0000 7f4ca2ffd700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4c980b2a10 0x7f4c980b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.991+0000 7f4c977fe700 1 -- 172.21.15.47:0/3490170135 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca401a410 con 0x55f112728ad0 2024-05-11T12:35:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.991+0000 7f4c977fe700 1 -- 172.21.15.47:0/3490170135 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca4019860 con 0x55f112728ad0 2024-05-11T12:35:11.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.991+0000 7f4c977fe700 1 -- 172.21.15.47:0/3490170135 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca40188c0 con 0x55f112728ad0 2024-05-11T12:35:11.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.993+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/3490170135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 msgr2=0x55f112728eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.993+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f112728eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400a800 tx=0x7f4ca40096d0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.993+0000 7f4ca2ffd700 1 -- 172.21.15.47:0/3605351937 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4c9c0a6ca0 con 0x7f4cac11cd30 2024-05-11T12:35:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.994+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/3490170135 shutdown_connections 2024-05-11T12:35:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.994+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112723ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.994+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f112728eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.994+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/3490170135 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f112723120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:11.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.994+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/3490170135 >> 172.21.15.47:0/3490170135 conn(0x55f11264fee0 msgr2=0x55f11271dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:12.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:11.998+0000 7f4cb19da700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4c980b2a10 0x7f4c980b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.003+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/3490170135 shutdown_connections 2024-05-11T12:35:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.004+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/3490170135 wait complete. 2024-05-11T12:35:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.005+0000 7f4cbe4d4b80 1 -- start start 2024-05-11T12:35:12.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.005+0000 7f4cbe4d4b80 1 Processor -- start 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.009+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112723660 0x55f11272f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.009+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112728ad0 0x55f11272c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.009+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f11272cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.009+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55f11272fe80 con 0x55f1127293f0 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.009+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55f1125dbcb0 con 0x55f112728ad0 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.009+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55f112730000 con 0x55f112723660 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.009+0000 7f4cbe4d4b80 10 client.0 _ll_register_callbacks cb 0x55f11260d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:12.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f11272cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f11272cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43648/0 (socket says 172.21.15.47:43648) 2024-05-11T12:35:12.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 -- 172.21.15.47:0/2277404427 learned_addr learned my addr 172.21.15.47:0/2277404427 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:12.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 -- 172.21.15.47:0/2277404427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112723660 msgr2=0x55f11272f1f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112723660 0x55f11272f1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 -- 172.21.15.47:0/2277404427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112728ad0 msgr2=0x55f11272c590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112728ad0 0x55f11272c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.010+0000 7f4cb21db700 1 -- 172.21.15.47:0/2277404427 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f112611710 con 0x55f1127293f0 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.011+0000 7f4cb19da700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4c980b2a10 0x7f4c980b4ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55f11271ebc0 tx=0x7f4ca800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.011+0000 7f4cb21db700 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f11272cad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0a7920 tx=0x7f4c9c0a4d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef33db0d2e18c041 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.011+0000 7f4c957fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.012+0000 7f4c957fa700 1 -- 172.21.15.47:0/2277404427 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0a6ea0 con 0x55f1127293f0 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.012+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca4019860 con 0x55f1127293f0 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.012+0000 7f4c957fa700 1 -- 172.21.15.47:0/2277404427 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4c9c0228f0 con 0x55f1127293f0 2024-05-11T12:35:12.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.016+0000 7f4c957fa700 1 -- 172.21.15.47:0/2277404427 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4c9c0a7040 con 0x55f1127293f0 2024-05-11T12:35:12.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.016+0000 7f4cbe4d4b80 10 client.15927 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:12.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.016+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f11274b660 con 0x55f1127293f0 2024-05-11T12:35:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4c957fa700 1 -- 172.21.15.47:0/2277404427 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca4019860 con 0x55f1127293f0 2024-05-11T12:35:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4c957fa700 1 -- 172.21.15.47:0/2277404427 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4c9c0a6350 con 0x55f1127293f0 2024-05-11T12:35:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4cbe4d4b80 10 client.15927 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4cbe4d4b80 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:12.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4cbe4d4b80 20 client.15927 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f11274dcf0 0x55f112750190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4cbe4d4b80 4 client.15927 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:35:12.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.019+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1126832c0 con 0x55f11274dcf0 2024-05-11T12:35:12.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.022+0000 7f4cb19da700 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f11274dcf0 0x55f112750190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.036+0000 7f4cb19da700 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f11274dcf0 0x55f112750190 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc43491d19e0ad74 server_cookie=e478546f1e00f193 in_seq=0 out_seq=0 2024-05-11T12:35:12.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.036+0000 7f4c957fa700 10 client.15927 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:12.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.042+0000 7f4c957fa700 1 -- 172.21.15.47:0/2277404427 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f1126832c0 con 0x55f11274dcf0 2024-05-11T12:35:12.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.042+0000 7f4c957fa700 10 client.15927 handle_command_reply: tid=0 2024-05-11T12:35:12.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.044+0000 7f4cbe4d4b80 1 client.15927 shutdown 2024-05-11T12:35:12.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.044+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 msgr2=0x55f11272cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:12.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.044+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f11272cad0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0a7920 tx=0x7f4c9c0a4d60 comp rx=0 tx=0).stop 2024-05-11T12:35:12.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.045+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 shutdown_connections 2024-05-11T12:35:12.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.045+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f11274dcf0 0x55f112750190 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.045+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f1127293f0 0x55f11272cad0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.045+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112723660 0x55f11272f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.045+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2277404427 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112728ad0 0x55f11272c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.045+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 >> 172.21.15.47:0/2277404427 conn(0x55f11264fee0 msgr2=0x55f112722110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:12.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.045+0000 7f4c957fa700 0 client.15927 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:12.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.048+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 shutdown_connections 2024-05-11T12:35:12.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.049+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2277404427 wait complete. 2024-05-11T12:35:12.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.049+0000 7f4cbe4d4b80 20 client.15927 trim_cache size 0 max 16384 2024-05-11T12:35:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.105+0000 7f4cbe4d4b80 1 Processor -- start 2024-05-11T12:35:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.105+0000 7f4cbe4d4b80 1 -- start start 2024-05-11T12:35:12.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.106+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f11264fee0 0x55f11272a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.106+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f11272eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.106+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f11272f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.106+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac117810 con 0x55f11264fee0 2024-05-11T12:35:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.106+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac11a090 con 0x55f112723660 2024-05-11T12:35:12.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.106+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac117e80 con 0x55f112728ad0 2024-05-11T12:35:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb11d9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f11272eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb11d9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f11272eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:53998/0 (socket says 172.21.15.47:53998) 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb11d9700 1 -- 172.21.15.47:0/1068695665 learned_addr learned my addr 172.21.15.47:0/1068695665 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb19da700 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f11264fee0 0x55f11272a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb19da700 1 -- 172.21.15.47:0/1068695665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 msgr2=0x55f11272f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb19da700 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f11272f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb19da700 1 -- 172.21.15.47:0/1068695665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 msgr2=0x55f11272eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb19da700 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f11272eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.107+0000 7f4cb19da700 1 -- 172.21.15.47:0/1068695665 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f112730000 con 0x55f11264fee0 2024-05-11T12:35:12.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.108+0000 7f4cb19da700 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f11264fee0 0x55f11272a110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55f1127509a0 tx=0x7f4ca8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de20793ad38019f server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.108+0000 7f4c96ffd700 1 -- 172.21.15.47:0/1068695665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca802b070 con 0x55f11264fee0 2024-05-11T12:35:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.109+0000 7f4c96ffd700 1 -- 172.21.15.47:0/1068695665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca8011cd0 con 0x55f11264fee0 2024-05-11T12:35:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.110+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/1068695665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f11264fee0 msgr2=0x55f11272a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.110+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f11264fee0 0x55f11272a110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55f1127509a0 tx=0x7f4ca8015000 comp rx=0 tx=0).stop 2024-05-11T12:35:12.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.110+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/1068695665 shutdown_connections 2024-05-11T12:35:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.110+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f112728ad0 0x55f11272f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.110+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f11264fee0 0x55f11272a110 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.110+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/1068695665 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f11272eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.110+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/1068695665 >> 172.21.15.47:0/1068695665 conn(0x55f11272c150 msgr2=0x55f112729a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.112+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/1068695665 shutdown_connections 2024-05-11T12:35:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.112+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/1068695665 wait complete. 2024-05-11T12:35:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.112+0000 7f4cbe4d4b80 1 -- start start 2024-05-11T12:35:12.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.113+0000 7f4cbe4d4b80 1 Processor -- start 2024-05-11T12:35:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.113+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f11264fee0 0x55f112735950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.113+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112732290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.113+0000 7f4cbe4d4b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f112728ad0 0x55f112732980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.114+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac118550 con 0x55f112728ad0 2024-05-11T12:35:12.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.114+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac1199c0 con 0x55f112723660 2024-05-11T12:35:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.114+0000 7f4cbe4d4b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac118c20 con 0x55f11264fee0 2024-05-11T12:35:12.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.114+0000 7f4cbe4d4b80 10 client.0 _ll_register_callbacks cb 0x55f11260d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:12.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.116+0000 7f4cb19da700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f11264fee0 0x55f112735950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.117+0000 7f4cb11d9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112732290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.117+0000 7f4cb21db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f112728ad0 0x55f112732980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.117+0000 7f4cb21db700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f112728ad0 0x55f112732980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:43670/0 (socket says 172.21.15.47:43670) 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.117+0000 7f4cb21db700 1 -- 172.21.15.47:0/2129279519 learned_addr learned my addr 172.21.15.47:0/2129279519 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.118+0000 7f4cb21db700 1 -- 172.21.15.47:0/2129279519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f11264fee0 msgr2=0x55f112735950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.118+0000 7f4cb21db700 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f11264fee0 0x55f112735950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.118+0000 7f4cb21db700 1 -- 172.21.15.47:0/2129279519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 msgr2=0x55f112732290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.118+0000 7f4cb21db700 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112732290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.118+0000 7f4cb21db700 1 -- 172.21.15.47:0/2129279519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1127b8620 con 0x55f112728ad0 2024-05-11T12:35:12.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.119+0000 7f4cb21db700 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f112728ad0 0x55f112732980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0aa530 tx=0x7f4c9c0a6bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e00ea39a449d19 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.121+0000 7f4c957fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.121+0000 7f4c957fa700 1 -- 172.21.15.47:0/2129279519 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c9c0a7210 con 0x55f112728ad0 2024-05-11T12:35:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.121+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca8011cd0 con 0x55f112728ad0 2024-05-11T12:35:12.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.121+0000 7f4c957fa700 1 -- 172.21.15.47:0/2129279519 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f4c9c0a9910 con 0x55f112728ad0 2024-05-11T12:35:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.127+0000 7f4c957fa700 1 -- 172.21.15.47:0/2129279519 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4c9c0a73b0 con 0x55f112728ad0 2024-05-11T12:35:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.128+0000 7f4cbe4d4b80 10 client.15933 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:12.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.128+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f1127b9800 con 0x55f112728ad0 2024-05-11T12:35:12.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.130+0000 7f4c957fa700 1 -- 172.21.15.47:0/2129279519 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca8011cd0 con 0x55f112728ad0 2024-05-11T12:35:12.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.130+0000 7f4c957fa700 1 -- 172.21.15.47:0/2129279519 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f4c9c0b36d0 con 0x55f112728ad0 2024-05-11T12:35:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.130+0000 7f4cbe4d4b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.130+0000 7f4cbe4d4b80 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:12.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.130+0000 7f4cbe4d4b80 20 client.15933 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.130+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f1127293f0 0x55f1127bbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.131+0000 7f4cbe4d4b80 4 client.15933 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:35:12.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.131+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f11267d360 con 0x55f1127293f0 2024-05-11T12:35:12.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.133+0000 7f4cb19da700 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f1127293f0 0x55f1127bbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.155+0000 7f4c957fa700 10 client.15933 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:12.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.155+0000 7f4cb19da700 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f1127293f0 0x55f1127bbd90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1868a7bc1c160b08 server_cookie=7ea774a34d192bec in_seq=0 out_seq=0 2024-05-11T12:35:12.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:12 smithi047 ceph-mon[30375]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 703 MiB/s wr, 1.63k op/s 2024-05-11T12:35:12.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:12 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:12 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:12 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:12 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.327+0000 7f4c957fa700 1 -- 172.21.15.47:0/2129279519 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f11267d360 con 0x55f1127293f0 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.327+0000 7f4c957fa700 10 client.15933 handle_command_reply: tid=0 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.327+0000 7f4cbe4d4b80 1 client.15933 shutdown 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.327+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f112728ad0 msgr2=0x55f112732980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.327+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f112728ad0 0x55f112732980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0aa530 tx=0x7f4c9c0a6bc0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 shutdown_connections 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55f1127293f0 0x55f1127bbd90 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55f112728ad0 0x55f112732980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55f11264fee0 0x55f112735950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 --2- 172.21.15.47:0/2129279519 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55f112723660 0x55f112732290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4c957fa700 0 client.15933 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:12.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 >> 172.21.15.47:0/2129279519 conn(0x55f11272c150 msgr2=0x55f112717e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 shutdown_connections 2024-05-11T12:35:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 1 -- 172.21.15.47:0/2129279519 wait complete. 2024-05-11T12:35:12.331 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:35:12.331 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:35:12.331 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "66f37004-4efb-4ca8-b107-a99c8cea34f8", 2024-05-11T12:35:12.331 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:35:12.331 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:35:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.328+0000 7f4cbe4d4b80 20 client.15933 trim_cache size 0 max 16384 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.334+0000 7f4ca0d70700 1 -- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4c980b2a10 msgr2=0x7f4c980b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.334+0000 7f4ca0d70700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4c980b2a10 0x7f4c980b4ed0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55f11271ebc0 tx=0x7f4ca800b5f0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.334+0000 7f4ca0d70700 1 -- 172.21.15.47:0/3605351937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac11cd30 msgr2=0x7f4cac1a8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.334+0000 7f4ca0d70700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac11cd30 0x7f4cac1a8980 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c00b2e0 tx=0x7f4c9c00b820 comp rx=0 tx=0).stop 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.335+0000 7f4ca0d70700 1 -- 172.21.15.47:0/3605351937 shutdown_connections 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.335+0000 7f4ca0d70700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f4cac11cd30 0x7f4cac1a8980 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.335+0000 7f4ca0d70700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f4c980b2a10 0x7f4c980b4ed0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.335+0000 7f4ca0d70700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4cac10aa90 0x7f4cac1ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.335+0000 7f4ca0d70700 1 --2- 172.21.15.47:0/3605351937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4cac111d30 0x7f4cac1aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.335+0000 7f4ca0d70700 1 -- 172.21.15.47:0/3605351937 >> 172.21.15.47:0/3605351937 conn(0x7f4cac06e6c0 msgr2=0x7f4cac11ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:12.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.336+0000 7f4ca0d70700 1 -- 172.21.15.47:0/3605351937 shutdown_connections 2024-05-11T12:35:12.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:12.336+0000 7f4ca0d70700 1 -- 172.21.15.47:0/3605351937 wait complete. 2024-05-11T12:35:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:12 smithi064 ceph-mon[34537]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 703 MiB/s wr, 1.63k op/s 2024-05-11T12:35:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:12 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:12 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:12 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:12 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:12 smithi050 ceph-mon[34072]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 703 MiB/s wr, 1.63k op/s 2024-05-11T12:35:12.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:12 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:12 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:12 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:12.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:12 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:13 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:35:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:13 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:13.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:13 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:35:13.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:13 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:35:13.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:13 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:13.366 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:13 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:35:13.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:13 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:35:13.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:13 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:13.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:13 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:35:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:14 smithi064 ceph-mon[34537]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 939 MiB/s wr, 1.84k op/s 2024-05-11T12:35:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:14 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:14.365 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:14 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:14.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:14 smithi050 ceph-mon[34072]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 939 MiB/s wr, 1.84k op/s 2024-05-11T12:35:14.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:14.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:14 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:14.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:14 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:14 smithi047 ceph-mon[30375]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 939 MiB/s wr, 1.84k op/s 2024-05-11T12:35:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:14 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:14.527 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:14 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:15.435 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:35:16.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:16 smithi064 ceph-mon[34537]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 711 MiB/s wr, 1.37k op/s 2024-05-11T12:35:16.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:16 smithi050 ceph-mon[34072]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 711 MiB/s wr, 1.37k op/s 2024-05-11T12:35:16.485 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:35:16.528 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:16 smithi047 ceph-mon[30375]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 711 MiB/s wr, 1.37k op/s 2024-05-11T12:35:17.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:35:17.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:35:17.329 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:35:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:35:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:35:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:35:17.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:35:18.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:18 smithi050 ceph-mon[34072]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 808 MiB/s wr, 1.71k op/s 2024-05-11T12:35:18.526 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:18 smithi047 ceph-mon[30375]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 808 MiB/s wr, 1.71k op/s 2024-05-11T12:35:18.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:18 smithi064 ceph-mon[34537]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 808 MiB/s wr, 1.71k op/s 2024-05-11T12:35:19.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:19 smithi047 ceph-mon[30375]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 812 MiB/s wr, 1.81k op/s 2024-05-11T12:35:19.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:19 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:35:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:19 smithi050 ceph-mon[34072]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 812 MiB/s wr, 1.81k op/s 2024-05-11T12:35:19.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:19 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:35:19.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:19 smithi064 ceph-mon[34537]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 812 MiB/s wr, 1.81k op/s 2024-05-11T12:35:19.614 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:19 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:35:21.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:21 smithi047 ceph-mon[30375]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 695 MiB/s wr, 1.45k op/s 2024-05-11T12:35:21.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:21 smithi064 ceph-mon[34537]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 695 MiB/s wr, 1.45k op/s 2024-05-11T12:35:21.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:21 smithi050 ceph-mon[34072]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 695 MiB/s wr, 1.45k op/s 2024-05-11T12:35:23.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.196+0000 7fc169f99700 1 -- 172.21.15.47:0/2419655301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164114150 msgr2=0x7fc1641165f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.196+0000 7fc169f99700 1 --2- 172.21.15.47:0/2419655301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164114150 0x7fc1641165f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0098f0 tx=0x7fc15c009c00 comp rx=0 tx=0).stop 2024-05-11T12:35:23.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.200+0000 7fc169f99700 1 -- 172.21.15.47:0/2419655301 shutdown_connections 2024-05-11T12:35:23.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.200+0000 7fc169f99700 1 --2- 172.21.15.47:0/2419655301 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164114150 0x7fc1641165f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.200+0000 7fc169f99700 1 --2- 172.21.15.47:0/2419655301 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc164101da0 0x7fc164102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.200+0000 7fc169f99700 1 --2- 172.21.15.47:0/2419655301 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164102750 0x7fc164113a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.200+0000 7fc169f99700 1 -- 172.21.15.47:0/2419655301 >> 172.21.15.47:0/2419655301 conn(0x7fc164078ac0 msgr2=0x7fc1640ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:23.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.206+0000 7fc169f99700 1 -- 172.21.15.47:0/2419655301 shutdown_connections 2024-05-11T12:35:23.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.206+0000 7fc169f99700 1 -- 172.21.15.47:0/2419655301 wait complete. 2024-05-11T12:35:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.208+0000 7fc169f99700 1 Processor -- start 2024-05-11T12:35:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.213+0000 7fc169f99700 1 -- start start 2024-05-11T12:35:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.214+0000 7fc169f99700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc164101da0 0x7fc164111530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.214+0000 7fc169f99700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164102750 0x7fc164111a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.214+0000 7fc169f99700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164114150 0x7fc16410a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.214+0000 7fc169f99700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc164119d60 con 0x7fc164114150 2024-05-11T12:35:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.214+0000 7fc169f99700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc164119be0 con 0x7fc164101da0 2024-05-11T12:35:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.214+0000 7fc169f99700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc164119ee0 con 0x7fc164102750 2024-05-11T12:35:23.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.218+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc164101da0 0x7fc164111530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.218+0000 7fc162ffd700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164102750 0x7fc164111a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.218+0000 7fc163fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164114150 0x7fc16410a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.218+0000 7fc163fff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164114150 0x7fc16410a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37602/0 (socket says 172.21.15.47:37602) 2024-05-11T12:35:23.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.218+0000 7fc163fff700 1 -- 172.21.15.47:0/3821217747 learned_addr learned my addr 172.21.15.47:0/3821217747 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:23.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.220+0000 7fc163fff700 1 -- 172.21.15.47:0/3821217747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164102750 msgr2=0x7fc164111a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.220+0000 7fc163fff700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164102750 0x7fc164111a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.220+0000 7fc163fff700 1 -- 172.21.15.47:0/3821217747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc164101da0 msgr2=0x7fc164111530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.220+0000 7fc163fff700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc164101da0 0x7fc164111530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.220+0000 7fc163fff700 1 -- 172.21.15.47:0/3821217747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16410b0a0 con 0x7fc164114150 2024-05-11T12:35:23.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.223+0000 7fc163fff700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164114150 0x7fc16410a7a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009810 tx=0x7fc15c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:23.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.225+0000 7fc160ff9700 1 -- 172.21.15.47:0/3821217747 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c003a90 con 0x7fc164114150 2024-05-11T12:35:23.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.225+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16410b330 con 0x7fc164114150 2024-05-11T12:35:23.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.225+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16410f230 con 0x7fc164114150 2024-05-11T12:35:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.235+0000 7fc17482fb80 1 Processor -- start 2024-05-11T12:35:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.235+0000 7fc160ff9700 1 -- 172.21.15.47:0/3821217747 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc15c003c30 con 0x7fc164114150 2024-05-11T12:35:23.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.236+0000 7fc17482fb80 1 -- start start 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38475ad0 0x55ac38475eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac384700a0 0x55ac38470520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac3847c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc17482fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ac383c2860 con 0x55ac38470a60 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc17482fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac383617c0 con 0x55ac384700a0 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc17482fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ac383be820 con 0x55ac38475ad0 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc160ff9700 1 -- 172.21.15.47:0/3821217747 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c017780 con 0x7fc164114150 2024-05-11T12:35:23.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.237+0000 7fc160ff9700 1 -- 172.21.15.47:0/3821217747 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc15c017a00 con 0x7fc164114150 2024-05-11T12:35:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.239+0000 7fc160ff9700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1500b29e0 0x7fc1500b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.239+0000 7fc163fff700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1500b29e0 0x7fc1500b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.240+0000 7fc160ff9700 1 -- 172.21.15.47:0/3821217747 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc15c0e4460 con 0x7fc164114150 2024-05-11T12:35:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.240+0000 7fc163fff700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1500b29e0 0x7fc1500b4ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ab760 tx=0x7fc15c0dafe0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:23.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.240+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac384700a0 0x55ac38470520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.240+0000 7fc162ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac3847c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.240+0000 7fc162ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac3847c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37616/0 (socket says 172.21.15.47:37616) 2024-05-11T12:35:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.240+0000 7fc162ffd700 1 -- 172.21.15.47:0/1502589300 learned_addr learned my addr 172.21.15.47:0/1502589300 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:23.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.240+0000 7fc162ffd700 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38475ad0 0x55ac38475eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.251+0000 7fc162ffd700 1 -- 172.21.15.47:0/1502589300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38475ad0 msgr2=0x55ac38475eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:23.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.251+0000 7fc162ffd700 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38475ad0 0x55ac38475eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.251+0000 7fc162ffd700 1 -- 172.21.15.47:0/1502589300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac384700a0 msgr2=0x55ac38470520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.251+0000 7fc162ffd700 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac384700a0 0x55ac38470520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.251+0000 7fc162ffd700 1 -- 172.21.15.47:0/1502589300 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3830f8d0 con 0x55ac38470a60 2024-05-11T12:35:23.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.261+0000 7fc162ffd700 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac3847c890 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc14c015470 tx=0x7fc14c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3c8143b5a54260 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.264+0000 7fc159ffb700 1 -- 172.21.15.47:0/1502589300 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c015c10 con 0x55ac38470a60 2024-05-11T12:35:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.264+0000 7fc159ffb700 1 -- 172.21.15.47:0/1502589300 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc14c015db0 con 0x55ac38470a60 2024-05-11T12:35:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.264+0000 7fc159ffb700 1 -- 172.21.15.47:0/1502589300 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c005660 con 0x55ac38470a60 2024-05-11T12:35:23.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.265+0000 7fc17482fb80 1 -- 172.21.15.47:0/1502589300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 msgr2=0x55ac3847c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.265+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac3847c890 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc14c015470 tx=0x7fc14c009310 comp rx=0 tx=0).stop 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.269+0000 7fc17482fb80 1 -- 172.21.15.47:0/1502589300 shutdown_connections 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.269+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac3847c890 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.269+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38475ad0 0x55ac38475eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.269+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1502589300 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac384700a0 0x55ac38470520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.269+0000 7fc17482fb80 1 -- 172.21.15.47:0/1502589300 >> 172.21.15.47:0/1502589300 conn(0x55ac3839ce60 msgr2=0x55ac3846a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.270+0000 7fc17482fb80 1 -- 172.21.15.47:0/1502589300 shutdown_connections 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.270+0000 7fc17482fb80 1 -- 172.21.15.47:0/1502589300 wait complete. 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.270+0000 7fc17482fb80 1 -- start start 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 1 Processor -- start 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac38479800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac38475ad0 0x55ac38479d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55ac3847ce30 con 0x55ac384700a0 2024-05-11T12:35:23.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x55ac38328c30 con 0x55ac38475ad0 2024-05-11T12:35:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x55ac3847cfb0 con 0x55ac38470a60 2024-05-11T12:35:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.271+0000 7fc17482fb80 10 client.0 _ll_register_callbacks cb 0x55ac3835a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.283+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac38479800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.284+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac38479800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:54048/0 (socket says 172.21.15.47:54048) 2024-05-11T12:35:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.284+0000 7fc1637fe700 1 -- 172.21.15.47:0/3067468493 learned_addr learned my addr 172.21.15.47:0/3067468493 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:23.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.284+0000 7fc162ffd700 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac38475ad0 0x55ac38479d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.284+0000 7fc162ffd700 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.286+0000 7fc162ffd700 1 -- 172.21.15.47:0/3067468493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 msgr2=0x55ac38479800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:23.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.286+0000 7fc162ffd700 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac38479800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.286+0000 7fc162ffd700 1 -- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac38475ad0 msgr2=0x55ac38479d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:23.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.286+0000 7fc162ffd700 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac38475ad0 0x55ac38479d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.286+0000 7fc162ffd700 1 -- 172.21.15.47:0/3067468493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3835e690 con 0x55ac384700a0 2024-05-11T12:35:23.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.286+0000 7fc162ffd700 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847c3f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc14c004fd0 tx=0x7fc14c0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8c0db261811aaa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:23.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.287+0000 7fc142ffd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:23.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.287+0000 7fc142ffd700 1 -- 172.21.15.47:0/3067468493 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc14c004950 con 0x55ac384700a0 2024-05-11T12:35:23.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.287+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc14c015db0 con 0x55ac384700a0 2024-05-11T12:35:23.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.287+0000 7fc142ffd700 1 -- 172.21.15.47:0/3067468493 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc14c004af0 con 0x55ac384700a0 2024-05-11T12:35:23.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.288+0000 7fc142ffd700 1 -- 172.21.15.47:0/3067468493 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc14c0058e0 con 0x55ac384700a0 2024-05-11T12:35:23.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.288+0000 7fc17482fb80 10 client.15945 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:23.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.288+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ac384a96b0 con 0x55ac384700a0 2024-05-11T12:35:23.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc142ffd700 1 -- 172.21.15.47:0/3067468493 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc14c015db0 con 0x55ac384700a0 2024-05-11T12:35:23.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc142ffd700 1 -- 172.21.15.47:0/3067468493 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc14c00c030 con 0x55ac384700a0 2024-05-11T12:35:23.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc17482fb80 10 client.15945 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:23.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc17482fb80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:23.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc17482fb80 20 client.15945 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:23.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc17482fb80 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac384ac0f0 0x55ac384ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc17482fb80 4 client.15945 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:35:23.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.289+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac383d0240 con 0x55ac384ac0f0 2024-05-11T12:35:23.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.293+0000 7fc1637fe700 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac384ac0f0 0x55ac384ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.299+0000 7fc1637fe700 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac384ac0f0 0x55ac384ae550 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7c72304067efb73 server_cookie=a0aebe9ede10e753 in_seq=0 out_seq=0 2024-05-11T12:35:23.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.300+0000 7fc142ffd700 10 client.15945 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:23.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.304+0000 7fc142ffd700 1 -- 172.21.15.47:0/3067468493 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac383d0240 con 0x55ac384ac0f0 2024-05-11T12:35:23.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.304+0000 7fc142ffd700 10 client.15945 handle_command_reply: tid=0 2024-05-11T12:35:23.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.304+0000 7fc17482fb80 1 client.15945 shutdown 2024-05-11T12:35:23.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.307+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 msgr2=0x55ac3847c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.307+0000 7fc17482fb80 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847c3f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc14c004fd0 tx=0x7fc14c0092a0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.308+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 shutdown_connections 2024-05-11T12:35:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.308+0000 7fc17482fb80 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac384ac0f0 0x55ac384ae550 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.308+0000 7fc17482fb80 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac38475ad0 0x55ac38479d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.308+0000 7fc17482fb80 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847c3f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.308+0000 7fc17482fb80 1 --2- 172.21.15.47:0/3067468493 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac38479800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.308+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 >> 172.21.15.47:0/3067468493 conn(0x55ac3839ce60 msgr2=0x55ac38465460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:23.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.308+0000 7fc142ffd700 0 client.15945 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:23.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.311+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 shutdown_connections 2024-05-11T12:35:23.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.311+0000 7fc17482fb80 1 -- 172.21.15.47:0/3067468493 wait complete. 2024-05-11T12:35:23.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.312+0000 7fc17482fb80 20 client.15945 trim_cache size 0 max 16384 2024-05-11T12:35:23.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.459+0000 7fc17482fb80 1 Processor -- start 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.472+0000 7fc17482fb80 1 -- start start 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.473+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 0x55ac38474c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.473+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac384700a0 0x55ac38485f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.473+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac38479d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.473+0000 7fc17482fb80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400b7e0 con 0x55ac38470a60 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.473+0000 7fc17482fb80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400e060 con 0x55ac3839ce60 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.473+0000 7fc17482fb80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400be50 con 0x55ac384700a0 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.474+0000 7fc162ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac38479d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.474+0000 7fc162ffd700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac38479d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37642/0 (socket says 172.21.15.47:37642) 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.474+0000 7fc162ffd700 1 -- 172.21.15.47:0/1379324876 learned_addr learned my addr 172.21.15.47:0/1379324876 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.474+0000 7fc1637fe700 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac384700a0 0x55ac38485f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.474+0000 7fc162ffd700 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 0x55ac38474c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.475+0000 7fc1637fe700 1 -- 172.21.15.47:0/1379324876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 msgr2=0x55ac38474c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.475+0000 7fc1637fe700 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 0x55ac38474c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.475+0000 7fc1637fe700 1 -- 172.21.15.47:0/1379324876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 msgr2=0x55ac38479d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.475+0000 7fc1637fe700 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac38479d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.475+0000 7fc1637fe700 1 -- 172.21.15.47:0/1379324876 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3847d000 con 0x55ac384700a0 2024-05-11T12:35:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.476+0000 7fc1637fe700 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac384700a0 0x55ac38485f50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b010 tx=0x7fc154009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e34ae70b0bae3b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.476+0000 7fc1597fa700 1 -- 172.21.15.47:0/1379324876 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154007c60 con 0x55ac384700a0 2024-05-11T12:35:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.477+0000 7fc1597fa700 1 -- 172.21.15.47:0/1379324876 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc154021030 con 0x55ac384700a0 2024-05-11T12:35:23.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.477+0000 7fc1597fa700 1 -- 172.21.15.47:0/1379324876 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15401d980 con 0x55ac384700a0 2024-05-11T12:35:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.478+0000 7fc17482fb80 1 -- 172.21.15.47:0/1379324876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac384700a0 msgr2=0x55ac38485f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.479+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac384700a0 0x55ac38485f50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc15400b010 tx=0x7fc154009040 comp rx=0 tx=0).stop 2024-05-11T12:35:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.479+0000 7fc17482fb80 1 -- 172.21.15.47:0/1379324876 shutdown_connections 2024-05-11T12:35:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.479+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac38470a60 0x55ac38479d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.479+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 0x55ac38474c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.479+0000 7fc17482fb80 1 --2- 172.21.15.47:0/1379324876 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac384700a0 0x55ac38485f50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.479+0000 7fc17482fb80 1 -- 172.21.15.47:0/1379324876 >> 172.21.15.47:0/1379324876 conn(0x55ac3846c970 msgr2=0x55ac3846f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:23.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.480+0000 7fc17482fb80 1 -- 172.21.15.47:0/1379324876 shutdown_connections 2024-05-11T12:35:23.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.480+0000 7fc17482fb80 1 -- 172.21.15.47:0/1379324876 wait complete. 2024-05-11T12:35:23.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.481+0000 7fc17482fb80 1 -- start start 2024-05-11T12:35:23.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.482+0000 7fc17482fb80 1 Processor -- start 2024-05-11T12:35:23.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.484+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 0x55ac38482870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.484+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.485+0000 7fc17482fb80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac3847b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.485+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.485+0000 7fc1637fe700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37652/0 (socket says 172.21.15.47:37652) 2024-05-11T12:35:23.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.486+0000 7fc1637fe700 1 -- 172.21.15.47:0/64307923 learned_addr learned my addr 172.21.15.47:0/64307923 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:23.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.486+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400c520 con 0x55ac384700a0 2024-05-11T12:35:23.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.487+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400d990 con 0x55ac3839ce60 2024-05-11T12:35:23.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.487+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc14400cbf0 con 0x55ac38470a60 2024-05-11T12:35:23.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.487+0000 7fc17482fb80 10 client.0 _ll_register_callbacks cb 0x55ac3835a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:23.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.488+0000 7fc1637fe700 1 -- 172.21.15.47:0/64307923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 msgr2=0x55ac3847b830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:35:23.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.488+0000 7fc1637fe700 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac3847b830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.488+0000 7fc1637fe700 1 -- 172.21.15.47:0/64307923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 msgr2=0x55ac38482870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:23.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.488+0000 7fc1637fe700 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 0x55ac38482870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.488+0000 7fc1637fe700 1 -- 172.21.15.47:0/64307923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3847c070 con 0x55ac384700a0 2024-05-11T12:35:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.490+0000 7fc1637fe700 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847b2d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc1540094f0 tx=0x7fc154007c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da207ea7ae4afed server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:23.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.490+0000 7fc142ffd700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:23.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.491+0000 7fc142ffd700 1 -- 172.21.15.47:0/64307923 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15402b060 con 0x55ac384700a0 2024-05-11T12:35:23.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.492+0000 7fc142ffd700 1 -- 172.21.15.47:0/64307923 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc15401d050 con 0x55ac384700a0 2024-05-11T12:35:23.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.492+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc154021030 con 0x55ac384700a0 2024-05-11T12:35:23.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.493+0000 7fc142ffd700 1 -- 172.21.15.47:0/64307923 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc15401d250 con 0x55ac384700a0 2024-05-11T12:35:23.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.495+0000 7fc17482fb80 10 client.15951 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:23.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.495+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ac38516a90 con 0x55ac384700a0 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.498+0000 7fc142ffd700 1 -- 172.21.15.47:0/64307923 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc154026470 con 0x55ac384700a0 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.498+0000 7fc142ffd700 1 -- 172.21.15.47:0/64307923 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fc154030460 con 0x55ac384700a0 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.498+0000 7fc17482fb80 10 client.15951 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.498+0000 7fc17482fb80 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.498+0000 7fc17482fb80 20 client.15951 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.498+0000 7fc17482fb80 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac38475ad0 0x55ac38519080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.499+0000 7fc17482fb80 4 client.15951 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:35:23.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.499+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc154021030 con 0x55ac38475ad0 2024-05-11T12:35:23.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.508+0000 7fc162ffd700 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac38475ad0 0x55ac38519080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:23.525 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:35:23.525 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:35:23.525 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:35:23.525 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:35:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.518+0000 7fc162ffd700 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac38475ad0 0x55ac38519080 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a6fcdb09a94ce00 server_cookie=ffb8b68670cf6ed2 in_seq=0 out_seq=0 2024-05-11T12:35:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.518+0000 7fc142ffd700 10 client.15951 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.520+0000 7fc142ffd700 1 -- 172.21.15.47:0/64307923 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc154021030 con 0x55ac38475ad0 2024-05-11T12:35:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.521+0000 7fc142ffd700 10 client.15951 handle_command_reply: tid=0 2024-05-11T12:35:23.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.521+0000 7fc17482fb80 1 client.15951 shutdown 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.521+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 msgr2=0x55ac3847b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.521+0000 7fc17482fb80 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847b2d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc1540094f0 tx=0x7fc154007c60 comp rx=0 tx=0).stop 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.521+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 shutdown_connections 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc17482fb80 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x55ac38475ad0 0x55ac38519080 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc17482fb80 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x55ac38470a60 0x55ac3847b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc17482fb80 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x55ac3839ce60 0x55ac38482870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc17482fb80 1 --2- 172.21.15.47:0/64307923 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55ac384700a0 0x55ac3847b2d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 >> 172.21.15.47:0/64307923 conn(0x55ac3846c970 msgr2=0x55ac3846f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc142ffd700 0 client.15951 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 shutdown_connections 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.522+0000 7fc17482fb80 1 -- 172.21.15.47:0/64307923 wait complete. 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.523+0000 7fc17482fb80 20 client.15951 trim_cache size 0 max 16384 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.524+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1500b29e0 msgr2=0x7fc1500b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.524+0000 7fc169f99700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1500b29e0 0x7fc1500b4ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0ab760 tx=0x7fc15c0dafe0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.524+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164114150 msgr2=0x7fc16410a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.524+0000 7fc169f99700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164114150 0x7fc16410a7a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009810 tx=0x7fc15c00fc60 comp rx=0 tx=0).stop 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.525+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 shutdown_connections 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.525+0000 7fc169f99700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc164114150 0x7fc16410a7a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.525+0000 7fc169f99700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc1500b29e0 0x7fc1500b4ea0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.525+0000 7fc169f99700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc164101da0 0x7fc164111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.525+0000 7fc169f99700 1 --2- 172.21.15.47:0/3821217747 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc164102750 0x7fc164111a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:23.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.525+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 >> 172.21.15.47:0/3821217747 conn(0x7fc164078ac0 msgr2=0x7fc1640fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:23.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.528+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 shutdown_connections 2024-05-11T12:35:23.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:23.529+0000 7fc169f99700 1 -- 172.21.15.47:0/3821217747 wait complete. 2024-05-11T12:35:23.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:23 smithi047 ceph-mon[30375]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 913 MiB/s wr, 1.70k op/s 2024-05-11T12:35:23.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:23 smithi064 ceph-mon[34537]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 913 MiB/s wr, 1.70k op/s 2024-05-11T12:35:23.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:23 smithi050 ceph-mon[34072]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 913 MiB/s wr, 1.70k op/s 2024-05-11T12:35:25.616 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:35:25.616 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:35:25.776 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:25 smithi047 ceph-mon[30375]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 673 MiB/s wr, 1.40k op/s 2024-05-11T12:35:25.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:25 smithi064 ceph-mon[34537]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 673 MiB/s wr, 1.40k op/s 2024-05-11T12:35:25.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:25 smithi050 ceph-mon[34072]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 673 MiB/s wr, 1.40k op/s 2024-05-11T12:35:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:35:27.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:27 smithi064 ceph-mon[34537]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 750 MiB/s wr, 1.72k op/s 2024-05-11T12:35:27.871 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:27 smithi047 ceph-mon[30375]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 750 MiB/s wr, 1.72k op/s 2024-05-11T12:35:27.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:27 smithi050 ceph-mon[34072]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 750 MiB/s wr, 1.72k op/s 2024-05-11T12:35:29.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:29 smithi064 ceph-mon[34537]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 762 MiB/s wr, 1.92k op/s 2024-05-11T12:35:29.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:29 smithi050 ceph-mon[34072]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 762 MiB/s wr, 1.92k op/s 2024-05-11T12:35:29.990 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:29 smithi047 ceph-mon[30375]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 762 MiB/s wr, 1.92k op/s 2024-05-11T12:35:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.427+0000 7fae1b3f5700 1 -- 172.21.15.47:0/843307395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 msgr2=0x7fae1411f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.427+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/843307395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 0x7fae1411f120 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fae0401a9a0 tx=0x7fae0401acb0 comp rx=0 tx=0).stop 2024-05-11T12:35:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.427+0000 7fae1b3f5700 1 -- 172.21.15.47:0/843307395 shutdown_connections 2024-05-11T12:35:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.427+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/843307395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 0x7fae1411f120 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:31.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.427+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/843307395 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae14111c40 0x7fae1410a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:31.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.427+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/843307395 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae141120b0 0x7fae1410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:31.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.427+0000 7fae1b3f5700 1 -- 172.21.15.47:0/843307395 >> 172.21.15.47:0/843307395 conn(0x7fae1406e530 msgr2=0x7fae1406e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:31.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.428+0000 7fae1b3f5700 1 -- 172.21.15.47:0/843307395 shutdown_connections 2024-05-11T12:35:31.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.428+0000 7fae1b3f5700 1 -- 172.21.15.47:0/843307395 wait complete. 2024-05-11T12:35:31.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 Processor -- start 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 -- start start 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae14111c40 0x7fae141185b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae141120b0 0x7fae14118af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 0x7fae141a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fae14114ce0 con 0x7fae1411cd30 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fae14114b60 con 0x7fae14111c40 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.430+0000 7fae1b3f5700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fae14114e60 con 0x7fae141120b0 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.433+0000 7fae1abf4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 0x7fae141a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.433+0000 7fae1abf4700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 0x7fae141a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58098/0 (socket says 172.21.15.47:58098) 2024-05-11T12:35:31.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.433+0000 7fae1abf4700 1 -- 172.21.15.47:0/183738637 learned_addr learned my addr 172.21.15.47:0/183738637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:31.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.434+0000 7fae19bf2700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae141120b0 0x7fae14118af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:31.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.435+0000 7fae19bf2700 1 -- 172.21.15.47:0/183738637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae14111c40 msgr2=0x7fae141185b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-11T12:35:31.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.435+0000 7fae19bf2700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae14111c40 0x7fae141185b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:31.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.435+0000 7fae19bf2700 1 -- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 msgr2=0x7fae141a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.435+0000 7fae19bf2700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 0x7fae141a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:31.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.435+0000 7fae19bf2700 1 -- 172.21.15.47:0/183738637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae141a52c0 con 0x7fae141120b0 2024-05-11T12:35:31.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.437+0000 7fae19bf2700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae141120b0 0x7fae14118af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae08006fd0 tx=0x7fae0800b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.437+0000 7fae137fe700 1 -- 172.21.15.47:0/183738637 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08015020 con 0x7fae141120b0 2024-05-11T12:35:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.438+0000 7fae137fe700 1 -- 172.21.15.47:0/183738637 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae08011ce0 con 0x7fae141120b0 2024-05-11T12:35:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.438+0000 7fae137fe700 1 -- 172.21.15.47:0/183738637 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08010d80 con 0x7fae141120b0 2024-05-11T12:35:31.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.438+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae141a55b0 con 0x7fae141120b0 2024-05-11T12:35:31.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.439+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae141ceea0 con 0x7fae141120b0 2024-05-11T12:35:31.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.450+0000 7fae117fa700 1 -- 172.21.15.47:0/183738637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1406f970 con 0x7fae141120b0 2024-05-11T12:35:31.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.455+0000 7fae137fe700 1 -- 172.21.15.47:0/183738637 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fae08018020 con 0x7fae141120b0 2024-05-11T12:35:31.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.464+0000 7fae137fe700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fadfc0b29c0 0x7fadfc0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:31.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.469+0000 7fae1a3f3700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fadfc0b29c0 0x7fadfc0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:31.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.472+0000 7fae137fe700 1 -- 172.21.15.47:0/183738637 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fae080e1a10 con 0x7fae141120b0 2024-05-11T12:35:31.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.472+0000 7fae137fe700 1 -- 172.21.15.47:0/183738637 <== mon.1 v2:172.21.15.64:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae0801f020 con 0x7fae141120b0 2024-05-11T12:35:31.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:31.476+0000 7fae1a3f3700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fadfc0b29c0 0x7fadfc0b4e80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00e350 tx=0x7fae0c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:31.777 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:31 smithi047 ceph-mon[30375]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 662 MiB/s wr, 1.61k op/s 2024-05-11T12:35:31.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:31 smithi064 ceph-mon[34537]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 662 MiB/s wr, 1.61k op/s 2024-05-11T12:35:31.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:31 smithi050 ceph-mon[34072]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 662 MiB/s wr, 1.61k op/s 2024-05-11T12:35:32.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.155+0000 7fae117fa700 1 -- 172.21.15.47:0/183738637 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae14119770 con 0x7fae141120b0 2024-05-11T12:35:32.162 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:35:32.162 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:35:32.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.158+0000 7fae137fe700 1 -- 172.21.15.47:0/183738637 <== mon.1 v2:172.21.15.64:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7fae080cb090 con 0x7fae141120b0 2024-05-11T12:35:32.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fadfc0b29c0 msgr2=0x7fadfc0b4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fadfc0b29c0 0x7fadfc0b4e80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00e350 tx=0x7fae0c00b040 comp rx=0 tx=0).stop 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae141120b0 msgr2=0x7fae14118af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae141120b0 0x7fae14118af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fae08006fd0 tx=0x7fae0800b7e0 comp rx=0 tx=0).stop 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 shutdown_connections 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fae1411cd30 0x7fae141a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fadfc0b29c0 0x7fadfc0b4e80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fae14111c40 0x7fae141185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.169+0000 7fae1b3f5700 1 --2- 172.21.15.47:0/183738637 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fae141120b0 0x7fae14118af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.170+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 >> 172.21.15.47:0/183738637 conn(0x7fae1406e530 msgr2=0x7fae14073f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:32.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.170+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 shutdown_connections 2024-05-11T12:35:32.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:32.170+0000 7fae1b3f5700 1 -- 172.21.15.47:0/183738637 wait complete. 2024-05-11T12:35:32.190 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:35:32.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:32 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/183738637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:35:32.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:32 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/183738637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:35:32.965 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:32 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/183738637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:35:33.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:33 smithi050 ceph-mon[34072]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.94k op/s 2024-05-11T12:35:34.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:33 smithi047 ceph-mon[30375]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.94k op/s 2024-05-11T12:35:34.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:33 smithi064 ceph-mon[34537]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 864 MiB/s wr, 1.94k op/s 2024-05-11T12:35:34.502 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:35:35.348 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:35:35.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:35 smithi050 ceph-mon[34072]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 670 MiB/s wr, 1.77k op/s 2024-05-11T12:35:36.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:35 smithi047 ceph-mon[30375]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 670 MiB/s wr, 1.77k op/s 2024-05-11T12:35:36.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:35 smithi064 ceph-mon[34537]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 670 MiB/s wr, 1.77k op/s 2024-05-11T12:35:37.916 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:37 smithi064 ceph-mon[34537]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 758 MiB/s wr, 2.03k op/s 2024-05-11T12:35:37.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:37 smithi050 ceph-mon[34072]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 758 MiB/s wr, 2.03k op/s 2024-05-11T12:35:38.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:37 smithi047 ceph-mon[30375]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 758 MiB/s wr, 2.03k op/s 2024-05-11T12:35:39.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:39 smithi050 ceph-mon[34072]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 795 MiB/s wr, 1.89k op/s 2024-05-11T12:35:39.954 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:39 smithi047 ceph-mon[30375]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 795 MiB/s wr, 1.89k op/s 2024-05-11T12:35:40.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:39 smithi064 ceph-mon[34537]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 795 MiB/s wr, 1.89k op/s 2024-05-11T12:35:40.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.547+0000 7f0410510700 1 -- 172.21.15.47:0/4262634705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408102450 msgr2=0x7f04081028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.547+0000 7f0410510700 1 --2- 172.21.15.47:0/4262634705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408102450 0x7f04081028d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f040000b0a0 tx=0x7f040000b3b0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.555+0000 7f0410510700 1 -- 172.21.15.47:0/4262634705 shutdown_connections 2024-05-11T12:35:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.555+0000 7f0410510700 1 --2- 172.21.15.47:0/4262634705 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0408102e10 0x7f040806b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.555+0000 7f0410510700 1 --2- 172.21.15.47:0/4262634705 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408108530 0x7f0408108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.555+0000 7f0410510700 1 --2- 172.21.15.47:0/4262634705 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408102450 0x7f04081028d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.555+0000 7f0410510700 1 -- 172.21.15.47:0/4262634705 >> 172.21.15.47:0/4262634705 conn(0x7f0408079400 msgr2=0x7f0408079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:40.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.560+0000 7f0410510700 1 -- 172.21.15.47:0/4262634705 shutdown_connections 2024-05-11T12:35:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.564+0000 7f0410510700 1 -- 172.21.15.47:0/4262634705 wait complete. 2024-05-11T12:35:40.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.565+0000 7f0410510700 1 Processor -- start 2024-05-11T12:35:40.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.577+0000 7f0410510700 1 -- start start 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.577+0000 7f0410510700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0408102450 0x7f040806e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.577+0000 7f0410510700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408102e10 0x7f040806eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.577+0000 7f0410510700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408108530 0x7f040806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.578+0000 7f0410510700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f040806d0f0 con 0x7f0408102450 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.578+0000 7f0410510700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f040806cf70 con 0x7f0408108530 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.578+0000 7f0410510700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f040806d270 con 0x7f0408102e10 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.579+0000 7f040daab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408102e10 0x7f040806eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.579+0000 7f040daab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408102e10 0x7f040806eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39648/0 (socket says 172.21.15.47:39648) 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.579+0000 7f040daab700 1 -- 172.21.15.47:0/724079055 learned_addr learned my addr 172.21.15.47:0/724079055 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.579+0000 7f040e2ac700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0408102450 0x7f040806e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.579+0000 7f040eaad700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408108530 0x7f040806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.580+0000 7f040daab700 1 -- 172.21.15.47:0/724079055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408108530 msgr2=0x7f040806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.580+0000 7f040daab700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408108530 0x7f040806f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.580+0000 7f040daab700 1 -- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0408102450 msgr2=0x7f040806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.580+0000 7f040daab700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0408102450 0x7f040806e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.580+0000 7f040daab700 1 -- 172.21.15.47:0/724079055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04080733c0 con 0x7f0408102e10 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.580+0000 7f040e2ac700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0408102450 0x7f040806e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.580+0000 7f040daab700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408102e10 0x7f040806eb70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f040000b070 tx=0x7f0400004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.582+0000 7f03ff7fe700 1 -- 172.21.15.47:0/724079055 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400043070 con 0x7f0408102e10 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.582+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0408073650 con 0x7f0408102e10 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.582+0000 7f03ff7fe700 1 -- 172.21.15.47:0/724079055 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0400007b80 con 0x7f0408102e10 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.582+0000 7f03ff7fe700 1 -- 172.21.15.47:0/724079055 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000052a0 con 0x7f0408102e10 2024-05-11T12:35:40.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.582+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0408073b90 con 0x7f0408102e10 2024-05-11T12:35:40.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.589+0000 7f041ada6b80 1 Processor -- start 2024-05-11T12:35:40.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.589+0000 7f041ada6b80 1 -- start start 2024-05-11T12:35:40.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 0x558133f08520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f08a60 0x558133f14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f041ada6b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558133e5a860 con 0x558133f080a0 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f041ada6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558133df97c0 con 0x558133f08a60 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f041ada6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558133e56820 con 0x558133f0dad0 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f040eaad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f040eaad700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39662/0 (socket says 172.21.15.47:39662) 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.590+0000 7f040eaad700 1 -- 172.21.15.47:0/3676231851 learned_addr learned my addr 172.21.15.47:0/3676231851 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.591+0000 7f040daab700 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f08a60 0x558133f14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.591+0000 7f040daab700 1 -- 172.21.15.47:0/3676231851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 msgr2=0x558133f0deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.591+0000 7f040daab700 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.591+0000 7f040daab700 1 -- 172.21.15.47:0/3676231851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 msgr2=0x558133f08520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.591+0000 7f040daab700 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 0x558133f08520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.591+0000 7f040daab700 1 -- 172.21.15.47:0/3676231851 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558133da78d0 con 0x558133f08a60 2024-05-11T12:35:40.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.591+0000 7f040daab700 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f08a60 0x558133f14890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f040004d000 tx=0x7f0400005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c626f5e33719985c server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.600+0000 7f03ff7fe700 1 -- 172.21.15.47:0/724079055 <== mon.1 v2:172.21.15.64:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f0400046070 con 0x7f0408102e10 2024-05-11T12:35:40.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.601+0000 7f03ff7fe700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f03f40b2a00 0x7f03f40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.603+0000 7f03ff7fe700 1 -- 172.21.15.47:0/724079055 <== mon.1 v2:172.21.15.64:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f040000f030 con 0x7f0408102e10 2024-05-11T12:35:40.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.603+0000 7f040eaad700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f03f40b2a00 0x7f03f40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.603+0000 7f03fcff9700 1 -- 172.21.15.47:0/3676231851 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000cc900 con 0x558133f08a60 2024-05-11T12:35:40.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.604+0000 7f03fcff9700 1 -- 172.21.15.47:0/3676231851 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0400052e50 con 0x558133f08a60 2024-05-11T12:35:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.605+0000 7f041ada6b80 1 -- 172.21.15.47:0/3676231851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f08a60 msgr2=0x558133f14890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.605+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f08a60 0x558133f14890 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f040004d000 tx=0x7f0400005a80 comp rx=0 tx=0).stop 2024-05-11T12:35:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.606+0000 7f041ada6b80 1 -- 172.21.15.47:0/3676231851 shutdown_connections 2024-05-11T12:35:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.606+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f08a60 0x558133f14890 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f040004d000 tx=0x7f0400005a80 comp rx=0 tx=0).stop 2024-05-11T12:35:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.606+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.606+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3676231851 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 0x558133f08520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.606+0000 7f041ada6b80 1 -- 172.21.15.47:0/3676231851 >> 172.21.15.47:0/3676231851 conn(0x558133e34e60 msgr2=0x558133f02990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:40.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.606+0000 7f041ada6b80 1 -- 172.21.15.47:0/3676231851 shutdown_connections 2024-05-11T12:35:40.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.611+0000 7f041ada6b80 1 -- 172.21.15.47:0/3676231851 wait complete. 2024-05-11T12:35:40.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.611+0000 7f041ada6b80 1 -- start start 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.612+0000 7f041ada6b80 1 Processor -- start 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.612+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 0x558133f162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.613+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f167f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.613+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f11920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.613+0000 7f041ada6b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x558133df97c0 con 0x558133f080a0 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.613+0000 7f041ada6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x558133dc0c30 con 0x558133f114a0 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.613+0000 7f041ada6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x558133f0e5e0 con 0x558133f0dad0 2024-05-11T12:35:40.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.613+0000 7f041ada6b80 10 client.0 _ll_register_callbacks cb 0x558133df2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:40.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.615+0000 7f040daab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f167f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.615+0000 7f040daab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f167f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39676/0 (socket says 172.21.15.47:39676) 2024-05-11T12:35:40.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.615+0000 7f040daab700 1 -- 172.21.15.47:0/1760119486 learned_addr learned my addr 172.21.15.47:0/1760119486 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:40.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.617+0000 7f040e2ac700 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 0x558133f162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.618+0000 7f040daab700 1 -- 172.21.15.47:0/1760119486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 msgr2=0x558133f11920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:40.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.618+0000 7f040daab700 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f11920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.618+0000 7f040daab700 1 -- 172.21.15.47:0/1760119486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 msgr2=0x558133f162b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:40.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.618+0000 7f040daab700 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 0x558133f162b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.618+0000 7f040daab700 1 -- 172.21.15.47:0/1760119486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558133df6690 con 0x558133f0dad0 2024-05-11T12:35:40.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.619+0000 7f040daab700 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f167f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04000ccc40 tx=0x7f0400009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aaacf8bfd95f5a8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:40.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.619+0000 7f03f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.64:3300/0 2024-05-11T12:35:40.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.620+0000 7f03f1ffb700 1 -- 172.21.15.47:0/1760119486 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000ccc70 con 0x558133f0dad0 2024-05-11T12:35:40.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.620+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558133e68240 con 0x558133f0dad0 2024-05-11T12:35:40.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.620+0000 7f03f1ffb700 1 -- 172.21.15.47:0/1760119486 <== mon.1 v2:172.21.15.64:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f04000cc3a0 con 0x558133f0dad0 2024-05-11T12:35:40.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.620+0000 7f03f1ffb700 1 -- 172.21.15.47:0/1760119486 <== mon.1 v2:172.21.15.64:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f04000d5020 con 0x558133f0dad0 2024-05-11T12:35:40.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.620+0000 7f041ada6b80 10 client.25537 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:40.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.621+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558133f1f400 con 0x558133f0dad0 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.621+0000 7f03f1ffb700 1 -- 172.21.15.47:0/1760119486 <== mon.1 v2:172.21.15.64:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558133e68240 con 0x558133f0dad0 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.622+0000 7f03f1ffb700 1 -- 172.21.15.47:0/1760119486 <== mon.1 v2:172.21.15.64:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f0400050060 con 0x558133f0dad0 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.622+0000 7f041ada6b80 10 client.25537 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.622+0000 7f041ada6b80 10 client.25537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.622+0000 7f041ada6b80 20 client.25537 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.622+0000 7f041ada6b80 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f21700 0x558133f23af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.622+0000 7f041ada6b80 4 client.25537 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:35:40.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.622+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558133e622e0 con 0x558133f21700 2024-05-11T12:35:40.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.623+0000 7f040e2ac700 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f21700 0x558133f23af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.629+0000 7f040e2ac700 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f21700 0x558133f23af0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b335b62249d361f server_cookie=5d8648fa402aae71 in_seq=0 out_seq=0 2024-05-11T12:35:40.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.630+0000 7f03f1ffb700 10 client.25537 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:40.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.631+0000 7f040eaad700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f03f40b2a00 0x7f03f40b4ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f03f8006fd0 tx=0x7f03f800bf30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:40.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.631+0000 7f03f1ffb700 1 -- 172.21.15.47:0/1760119486 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558133e622e0 con 0x558133f21700 2024-05-11T12:35:40.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.631+0000 7f03f1ffb700 10 client.25537 handle_command_reply: tid=0 2024-05-11T12:35:40.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.631+0000 7f041ada6b80 1 client.25537 shutdown 2024-05-11T12:35:40.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.637+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 msgr2=0x558133f167f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.637+0000 7f041ada6b80 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f167f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f04000ccc40 tx=0x7f0400009d20 comp rx=0 tx=0).stop 2024-05-11T12:35:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.644+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 shutdown_connections 2024-05-11T12:35:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.644+0000 7f041ada6b80 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f21700 0x558133f23af0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.644+0000 7f041ada6b80 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f11920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.644+0000 7f041ada6b80 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f080a0 0x558133f162b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.644+0000 7f041ada6b80 1 --2- 172.21.15.47:0/1760119486 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f167f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.644+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 >> 172.21.15.47:0/1760119486 conn(0x558133e34e60 msgr2=0x558133f07090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:40.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.644+0000 7f03f1ffb700 0 client.25537 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.647+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 shutdown_connections 2024-05-11T12:35:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.647+0000 7f041ada6b80 1 -- 172.21.15.47:0/1760119486 wait complete. 2024-05-11T12:35:40.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.647+0000 7f041ada6b80 20 client.25537 trim_cache size 0 max 16384 2024-05-11T12:35:40.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.698+0000 7f041ada6b80 1 Processor -- start 2024-05-11T12:35:40.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.700+0000 7f041ada6b80 1 -- start start 2024-05-11T12:35:40.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.701+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f080a0 0x558133f0bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.701+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.701+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f114a0 0x558133f293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.701+0000 7f041ada6b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0408116690 con 0x558133f114a0 2024-05-11T12:35:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.701+0000 7f041ada6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0408118f10 con 0x558133f080a0 2024-05-11T12:35:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.701+0000 7f041ada6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0408116d00 con 0x558133f0dad0 2024-05-11T12:35:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.702+0000 7f040daab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.702+0000 7f040daab700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.64:3300/0 says I am v2:172.21.15.47:39680/0 (socket says 172.21.15.47:39680) 2024-05-11T12:35:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.702+0000 7f040daab700 1 -- 172.21.15.47:0/3886034088 learned_addr learned my addr 172.21.15.47:0/3886034088 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.704+0000 7f040daab700 1 -- 172.21.15.47:0/3886034088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f080a0 msgr2=0x558133f0bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.704+0000 7f040daab700 1 --2- 172.21.15.47:0/3886034088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f080a0 0x558133f0bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.704+0000 7f040daab700 1 -- 172.21.15.47:0/3886034088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f114a0 msgr2=0x558133f293b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.704+0000 7f040daab700 1 --2- 172.21.15.47:0/3886034088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f114a0 0x558133f293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.704+0000 7f040daab700 1 -- 172.21.15.47:0/3886034088 --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558133f045f0 con 0x558133f0dad0 2024-05-11T12:35:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.705+0000 7f040daab700 1 --2- 172.21.15.47:0/3886034088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0ee70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04000e6040 tx=0x7f0400005a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfde6ff2c998873e server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.706+0000 7f03f3fff700 1 -- 172.21.15.47:0/3886034088 <== mon.1 v2:172.21.15.64:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000cce10 con 0x558133f0dad0 2024-05-11T12:35:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.706+0000 7f03f3fff700 1 -- 172.21.15.47:0/3886034088 <== mon.1 v2:172.21.15.64:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040007b070 con 0x558133f0dad0 2024-05-11T12:35:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.707+0000 7f041ada6b80 1 -- 172.21.15.47:0/3886034088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 msgr2=0x558133f0ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.707+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3886034088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0ee70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f04000e6040 tx=0x7f0400005a80 comp rx=0 tx=0).stop 2024-05-11T12:35:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.708+0000 7f041ada6b80 1 -- 172.21.15.47:0/3886034088 shutdown_connections 2024-05-11T12:35:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.708+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3886034088 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f114a0 0x558133f293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.708+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3886034088 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f080a0 0x558133f0bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.708+0000 7f041ada6b80 1 --2- 172.21.15.47:0/3886034088 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f0dad0 0x558133f0ee70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.708+0000 7f041ada6b80 1 -- 172.21.15.47:0/3886034088 >> 172.21.15.47:0/3886034088 conn(0x558133e34e60 msgr2=0x558133f0e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.718+0000 7f041ada6b80 1 -- 172.21.15.47:0/3886034088 shutdown_connections 2024-05-11T12:35:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.718+0000 7f041ada6b80 1 -- 172.21.15.47:0/3886034088 wait complete. 2024-05-11T12:35:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.719+0000 7f041ada6b80 1 -- start start 2024-05-11T12:35:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.719+0000 7f041ada6b80 1 Processor -- start 2024-05-11T12:35:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.720+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f080a0 0x558133f1d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.720+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f0dad0 0x558133f18ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.720+0000 7f041ada6b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f19430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.720+0000 7f041ada6b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04081173d0 con 0x558133f0dad0 2024-05-11T12:35:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.720+0000 7f041ada6b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f0408118840 con 0x558133f114a0 2024-05-11T12:35:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.720+0000 7f041ada6b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f0408117aa0 con 0x558133f080a0 2024-05-11T12:35:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.720+0000 7f041ada6b80 10 client.0 _ll_register_callbacks cb 0x558133df2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.726+0000 7f040eaad700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f19430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.726+0000 7f040e2ac700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f080a0 0x558133f1d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.726+0000 7f040eaad700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f19430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:38030/0 (socket says 172.21.15.47:38030) 2024-05-11T12:35:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.727+0000 7f040daab700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f0dad0 0x558133f18ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.727+0000 7f040eaad700 1 -- 172.21.15.47:0/2269585792 learned_addr learned my addr 172.21.15.47:0/2269585792 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.734+0000 7f040eaad700 1 -- 172.21.15.47:0/2269585792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f080a0 msgr2=0x558133f1d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.734+0000 7f040eaad700 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f080a0 0x558133f1d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.734+0000 7f040eaad700 1 -- 172.21.15.47:0/2269585792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f0dad0 msgr2=0x558133f18ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.734+0000 7f040eaad700 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f0dad0 0x558133f18ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.734+0000 7f040eaad700 1 -- 172.21.15.47:0/2269585792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558133f19970 con 0x558133f114a0 2024-05-11T12:35:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.738+0000 7f040eaad700 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f19430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f03f8012ed0 tx=0x7f03f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1544c2f455a3cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:40.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.739+0000 7f03f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.50:3300/0 2024-05-11T12:35:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.739+0000 7f03f1ffb700 1 -- 172.21.15.47:0/2269585792 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8015cf0 con 0x558133f114a0 2024-05-11T12:35:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.739+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040007b070 con 0x558133f114a0 2024-05-11T12:35:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.739+0000 7f03f1ffb700 1 -- 172.21.15.47:0/2269585792 <== mon.2 v2:172.21.15.50:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f03f8022430 con 0x558133f114a0 2024-05-11T12:35:40.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.740+0000 7f03f1ffb700 1 -- 172.21.15.47:0/2269585792 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f03f8022630 con 0x558133f114a0 2024-05-11T12:35:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.743+0000 7f041ada6b80 10 client.25520 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:40.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.743+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558133f8bd10 con 0x558133f114a0 2024-05-11T12:35:40.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.744+0000 7f03f1ffb700 1 -- 172.21.15.47:0/2269585792 <== mon.2 v2:172.21.15.50:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040007b070 con 0x558133f114a0 2024-05-11T12:35:40.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.744+0000 7f03f1ffb700 1 -- 172.21.15.47:0/2269585792 <== mon.2 v2:172.21.15.50:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f03f802a030 con 0x558133f114a0 2024-05-11T12:35:40.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.747+0000 7f041ada6b80 10 client.25520 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:40.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.747+0000 7f041ada6b80 10 client.25520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:40.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.747+0000 7f041ada6b80 20 client.25520 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:40.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.747+0000 7f041ada6b80 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f8c350 0x558133f8e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:40.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.747+0000 7f041ada6b80 4 client.25520 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:35:40.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.747+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558133cda250 con 0x558133f8c350 2024-05-11T12:35:40.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.751+0000 7f040e2ac700 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f8c350 0x558133f8e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.760+0000 7f040e2ac700 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f8c350 0x558133f8e770 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cfb689c6735a84d server_cookie=6307ef0aaf9f265c in_seq=0 out_seq=0 2024-05-11T12:35:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.763+0000 7f03f1ffb700 10 client.25520 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.763+0000 7f03f1ffb700 1 -- 172.21.15.47:0/2269585792 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558133cda250 con 0x558133f8c350 2024-05-11T12:35:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.763+0000 7f03f1ffb700 10 client.25520 handle_command_reply: tid=0 2024-05-11T12:35:40.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.763+0000 7f041ada6b80 1 client.25520 shutdown 2024-05-11T12:35:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.765+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 msgr2=0x558133f19430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.765+0000 7f041ada6b80 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f19430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f03f8012ed0 tx=0x7f03f8004750 comp rx=0 tx=0).stop 2024-05-11T12:35:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.766+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 shutdown_connections 2024-05-11T12:35:40.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.766+0000 7f041ada6b80 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x558133f8c350 0x558133f8e770 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.766+0000 7f041ada6b80 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x558133f114a0 0x558133f19430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.766+0000 7f041ada6b80 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x558133f080a0 0x558133f1d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.766+0000 7f041ada6b80 1 --2- 172.21.15.47:0/2269585792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x558133f0dad0 0x558133f18ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.766+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 >> 172.21.15.47:0/2269585792 conn(0x558133e34e60 msgr2=0x558133f07920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:40.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.766+0000 7f03f1ffb700 0 client.25520 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:40.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.768+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 shutdown_connections 2024-05-11T12:35:40.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.772+0000 7f041ada6b80 1 -- 172.21.15.47:0/2269585792 wait complete. 2024-05-11T12:35:40.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.772+0000 7f041ada6b80 20 client.25520 trim_cache size 0 max 16384 2024-05-11T12:35:40.773 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:35:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.780+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f03f40b2a00 msgr2=0x7f03f40b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.780+0000 7f0410510700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f03f40b2a00 0x7f03f40b4ec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f03f8006fd0 tx=0x7f03f800bf30 comp rx=0 tx=0).stop 2024-05-11T12:35:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.780+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408102e10 msgr2=0x7f040806eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:40.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.780+0000 7f0410510700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408102e10 0x7f040806eb70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f040000b070 tx=0x7f0400004d60 comp rx=0 tx=0).stop 2024-05-11T12:35:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.784+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 shutdown_connections 2024-05-11T12:35:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.784+0000 7f0410510700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f03f40b2a00 0x7f03f40b4ec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.784+0000 7f0410510700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0408108530 0x7f040806f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.785+0000 7f0410510700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0408102450 0x7f040806e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.785+0000 7f0410510700 1 --2- 172.21.15.47:0/724079055 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f0408102e10 0x7f040806eb70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:40.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.785+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 >> 172.21.15.47:0/724079055 conn(0x7f0408079400 msgr2=0x7f0408101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:40.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.789+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 shutdown_connections 2024-05-11T12:35:40.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:40.792+0000 7f0410510700 1 -- 172.21.15.47:0/724079055 wait complete. 2024-05-11T12:35:42.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:41 smithi047 ceph-mon[30375]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 694 MiB/s wr, 1.41k op/s 2024-05-11T12:35:42.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:41 smithi064 ceph-mon[34537]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 694 MiB/s wr, 1.41k op/s 2024-05-11T12:35:42.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:41 smithi050 ceph-mon[34072]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 694 MiB/s wr, 1.41k op/s 2024-05-11T12:35:43.735 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-11T12:35:43.736 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-11T12:35:44.030 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:43 smithi047 ceph-mon[30375]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 MiB/s wr, 1.87k op/s 2024-05-11T12:35:44.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:43 smithi064 ceph-mon[34537]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 MiB/s wr, 1.87k op/s 2024-05-11T12:35:44.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:43 smithi050 ceph-mon[34072]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 MiB/s wr, 1.87k op/s 2024-05-11T12:35:44.872 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:35:46.028 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:45 smithi047 ceph-mon[30375]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 702 MiB/s wr, 1.62k op/s 2024-05-11T12:35:46.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:45 smithi064 ceph-mon[34537]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 702 MiB/s wr, 1.62k op/s 2024-05-11T12:35:46.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:45 smithi050 ceph-mon[34072]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 702 MiB/s wr, 1.62k op/s 2024-05-11T12:35:47.156 INFO:tasks.workunit.client.0.smithi047.stdout:stopping iogen 2024-05-11T12:35:47.157 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo 'stopping iogen' 2024-05-11T12:35:47.157 INFO:tasks.workunit.client.0.smithi047.stderr:+ ./iogen -k 2024-05-11T12:35:47.247 INFO:tasks.workunit.client.0.smithi047.stdout:OK 2024-05-11T12:35:47.248 INFO:tasks.workunit.client.0.smithi047.stderr:+ echo OK 2024-05-11T12:35:48.027 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:47 smithi047 ceph-mon[30375]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 737 MiB/s wr, 1.63k op/s 2024-05-11T12:35:48.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:47 smithi064 ceph-mon[34537]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 737 MiB/s wr, 1.63k op/s 2024-05-11T12:35:48.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:47 smithi050 ceph-mon[34072]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 737 MiB/s wr, 1.63k op/s 2024-05-11T12:35:49.576 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-11T12:35:49.576 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-11T12:35:49.869 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-05-11T12:35:49.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-11T12:35:49.892 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:49 smithi047 ceph-mon[30375]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 825 MiB/s wr, 1.66k op/s 2024-05-11T12:35:50.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:49 smithi064 ceph-mon[34537]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 825 MiB/s wr, 1.66k op/s 2024-05-11T12:35:50.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:49 smithi050 ceph-mon[34072]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 825 MiB/s wr, 1.66k op/s 2024-05-11T12:35:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 -- 172.21.15.47:0/1245848799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818103800 msgr2=0x7ff818103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 --2- 172.21.15.47:0/1245848799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818103800 0x7ff818103be0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff8140099c0 tx=0x7ff814009cd0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 -- 172.21.15.47:0/1245848799 shutdown_connections 2024-05-11T12:35:50.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 --2- 172.21.15.47:0/1245848799 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff818108900 0x7ff818110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 --2- 172.21.15.47:0/1245848799 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818103800 0x7ff818103be0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 --2- 172.21.15.47:0/1245848799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 0x7ff8181081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 -- 172.21.15.47:0/1245848799 >> 172.21.15.47:0/1245848799 conn(0x7ff81807a6b0 msgr2=0x7ff81807aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 -- 172.21.15.47:0/1245848799 shutdown_connections 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.322+0000 7ff82057b700 1 -- 172.21.15.47:0/1245848799 wait complete. 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 Processor -- start 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 -- start start 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff818103800 0x7ff81819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 0x7ff81819fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818108900 0x7ff818199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff818113750 con 0x7ff818108900 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181135d0 con 0x7ff8181041b0 2024-05-11T12:35:50.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff82057b700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181138d0 con 0x7ff818103800 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 0x7ff81819fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 0x7ff81819fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:36744/0 (socket says 172.21.15.47:36744) 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 -- 172.21.15.47:0/3952680658 learned_addr learned my addr 172.21.15.47:0/3952680658 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81eb18700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818108900 0x7ff818199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81e317700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff818103800 0x7ff81819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 -- 172.21.15.47:0/3952680658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff818103800 msgr2=0x7ff81819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff818103800 0x7ff81819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 -- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818108900 msgr2=0x7ff818199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818108900 0x7ff818199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81db16700 1 -- 172.21.15.47:0/3952680658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81819a170 con 0x7ff8181041b0 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.323+0000 7ff81eb18700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818108900 0x7ff818199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.324+0000 7ff81db16700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 0x7ff81819fd90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c920 tx=0x7ff80800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:50.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.324+0000 7ff80f7fe700 1 -- 172.21.15.47:0/3952680658 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808004a30 con 0x7ff8181041b0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.324+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81819a460 con 0x7ff8181041b0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.324+0000 7ff80f7fe700 1 -- 172.21.15.47:0/3952680658 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8080054d0 con 0x7ff8181041b0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.324+0000 7ff80f7fe700 1 -- 172.21.15.47:0/3952680658 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808007bd0 con 0x7ff8181041b0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.324+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8181a6720 con 0x7ff8181041b0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.327+0000 7ff80f7fe700 1 -- 172.21.15.47:0/3952680658 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff808004bd0 con 0x7ff8181041b0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.328+0000 7ff80f7fe700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8040b2880 0x7ff8040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.328+0000 7ff82ae11b80 1 Processor -- start 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.328+0000 7ff81e317700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8040b2880 0x7ff8040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.328+0000 7ff80f7fe700 1 -- 172.21.15.47:0/3952680658 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff8080e1df0 con 0x7ff8181041b0 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.328+0000 7ff82ae11b80 1 -- start start 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742290 0x5641d6742670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d673c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d673cfc0 0x5641d673d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5641d668a9c0 con 0x5641d6742bb0 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5641d65efe20 con 0x5641d6742290 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5641d6686980 con 0x5641d673cfc0 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8040b2880 0x7ff8040b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff81819ab50 tx=0x7ff81400b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742290 0x5641d6742670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81eb18700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d673cfc0 0x5641d673d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742290 0x5641d6742670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:36756/0 (socket says 172.21.15.47:36756) 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 -- 172.21.15.47:0/669021782 learned_addr learned my addr 172.21.15.47:0/669021782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81db16700 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d673c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 -- 172.21.15.47:0/669021782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d673cfc0 msgr2=0x5641d673d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d673cfc0 0x5641d673d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.394 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:35:50.394 INFO:teuthology.orchestra.run.smithi047.stdout: "return_code": 0, 2024-05-11T12:35:50.394 INFO:teuthology.orchestra.run.smithi047.stdout: "scrub_tag": "730f425a-a60a-44c6-8f2b-35abb1650bf1", 2024-05-11T12:35:50.394 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": "asynchronous" 2024-05-11T12:35:50.394 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 -- 172.21.15.47:0/669021782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 msgr2=0x5641d673c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d673c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 -- 172.21.15.47:0/669021782 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641d65d7950 con 0x5641d6742290 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81db16700 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d673c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.329+0000 7ff81e317700 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742290 0x5641d6742670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff814003800 tx=0x7ff814020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96803f96a19ecaec server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff80cff9700 1 -- 172.21.15.47:0/669021782 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8140249f0 con 0x5641d6742290 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff80cff9700 1 -- 172.21.15.47:0/669021782 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814024b90 con 0x5641d6742290 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff80cff9700 1 -- 172.21.15.47:0/669021782 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8140306b0 con 0x5641d6742290 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 -- 172.21.15.47:0/669021782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742290 msgr2=0x5641d6742670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742290 0x5641d6742670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff814003800 tx=0x7ff814020000 comp rx=0 tx=0).stop 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 -- 172.21.15.47:0/669021782 shutdown_connections 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d673cfc0 0x5641d673d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742290 0x5641d6742670 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/669021782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d673c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 -- 172.21.15.47:0/669021782 >> 172.21.15.47:0/669021782 conn(0x5641d6664bf0 msgr2=0x5641d6733060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.330+0000 7ff82ae11b80 1 -- 172.21.15.47:0/669021782 shutdown_connections 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 -- 172.21.15.47:0/669021782 wait complete. 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 -- start start 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 Processor -- start 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6748950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6745cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d6746230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5641d6749670 con 0x5641d673cfc0 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5641d6643f50 con 0x5641d6742bb0 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5641d67497f0 con 0x5641d6742290 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff81e317700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6748950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff82ae11b80 10 client.0 _ll_register_callbacks cb 0x5641d66229d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:50.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff81e317700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6748950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60488/0 (socket says 172.21.15.47:60488) 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff81e317700 1 -- 172.21.15.47:0/4082087967 learned_addr learned my addr 172.21.15.47:0/4082087967 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.331+0000 7ff81db16700 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6745cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff81eb18700 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d6746230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff81e317700 1 -- 172.21.15.47:0/4082087967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 msgr2=0x5641d6745cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff81e317700 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6745cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff81e317700 1 -- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 msgr2=0x5641d6746230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff81e317700 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d6746230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff81e317700 1 -- 172.21.15.47:0/4082087967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641d66417e0 con 0x5641d673cfc0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff81e317700 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6748950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f50 tx=0x7ff814024870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c68fcb8efac1745 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff801ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff801ffb700 1 -- 172.21.15.47:0/4082087967 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400f720 con 0x5641d673cfc0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff814024b90 con 0x5641d673cfc0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff801ffb700 1 -- 172.21.15.47:0/4082087967 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff81400f8c0 con 0x5641d673cfc0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff801ffb700 1 -- 172.21.15.47:0/4082087967 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff814030450 con 0x5641d673cfc0 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff82ae11b80 10 client.15975 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:50.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5641d6749670 con 0x5641d673cfc0 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.332+0000 7ff801ffb700 1 -- 172.21.15.47:0/4082087967 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814024b90 con 0x5641d673cfc0 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff801ffb700 1 -- 172.21.15.47:0/4082087967 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff814038530 con 0x5641d673cfc0 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff82ae11b80 10 client.15975 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff82ae11b80 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff82ae11b80 20 client.15975 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d6756220 0x5641d6758610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff82ae11b80 4 client.15975 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641d66983a0 con 0x5641d6756220 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff81db16700 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d6756220 0x5641d6758610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff801ffb700 10 client.15975 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.333+0000 7ff81db16700 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d6756220 0x5641d6758610 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb778c7e5d772c25 server_cookie=20295d9bf6dc45bc in_seq=0 out_seq=0 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff801ffb700 1 -- 172.21.15.47:0/4082087967 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5641d66983a0 con 0x5641d6756220 2024-05-11T12:35:50.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff801ffb700 10 client.15975 handle_command_reply: tid=0 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 client.15975 shutdown 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 msgr2=0x5641d6748950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6748950 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7ff814009f50 tx=0x7ff814024870 comp rx=0 tx=0).stop 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 shutdown_connections 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d6756220 0x5641d6758610 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d6746230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6748950 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/4082087967 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6745cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff801ffb700 0 client.15975 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 >> 172.21.15.47:0/4082087967 conn(0x5641d6664bf0 msgr2=0x5641d672d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 shutdown_connections 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 1 -- 172.21.15.47:0/4082087967 wait complete. 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.334+0000 7ff82ae11b80 20 client.15975 trim_cache size 0 max 16384 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 Processor -- start 2024-05-11T12:35:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 -- start start 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6752700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d672d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d67472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181a9c90 con 0x5641d673cfc0 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181ac4f0 con 0x5641d6742bb0 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181aa2e0 con 0x5641d6742290 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff81e317700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6752700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff81e317700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6752700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60492/0 (socket says 172.21.15.47:60492) 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff81e317700 1 -- 172.21.15.47:0/2558959952 learned_addr learned my addr 172.21.15.47:0/2558959952 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.356+0000 7ff81eb18700 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d67472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff81db16700 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d672d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff81e317700 1 -- 172.21.15.47:0/2558959952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 msgr2=0x5641d672d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff81e317700 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d672d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff81e317700 1 -- 172.21.15.47:0/2558959952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 msgr2=0x5641d67472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff81e317700 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d67472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff81e317700 1 -- 172.21.15.47:0/2558959952 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641d6748590 con 0x5641d673cfc0 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff81e317700 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6752700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff814006010 tx=0x7ff8140096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cce4a9263fffa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff803fff700 1 -- 172.21.15.47:0/2558959952 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814045070 con 0x5641d673cfc0 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff803fff700 1 -- 172.21.15.47:0/2558959952 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814038410 con 0x5641d673cfc0 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.357+0000 7ff803fff700 1 -- 172.21.15.47:0/2558959952 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8140305f0 con 0x5641d673cfc0 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 -- 172.21.15.47:0/2558959952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 msgr2=0x5641d6752700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6752700 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff814006010 tx=0x7ff8140096d0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 -- 172.21.15.47:0/2558959952 shutdown_connections 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d6742bb0 0x5641d67472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d673cfc0 0x5641d6752700 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/2558959952 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d672d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 -- 172.21.15.47:0/2558959952 >> 172.21.15.47:0/2558959952 conn(0x5641d6664bf0 msgr2=0x5641d67416d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 -- 172.21.15.47:0/2558959952 shutdown_connections 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 -- 172.21.15.47:0/2558959952 wait complete. 2024-05-11T12:35:50.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.358+0000 7ff82ae11b80 1 -- start start 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 1 Processor -- start 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d673cfc0 0x5641d674f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6749a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d6749f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181aa9b0 con 0x5641d6742bb0 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181abe20 con 0x5641d673cfc0 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7ff8181ab080 con 0x5641d6742290 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff81eb18700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d6749f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff82ae11b80 10 client.0 _ll_register_callbacks cb 0x5641d66229d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff81eb18700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d6749f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60498/0 (socket says 172.21.15.47:60498) 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.359+0000 7ff81eb18700 1 -- 172.21.15.47:0/3181528870 learned_addr learned my addr 172.21.15.47:0/3181528870 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81e317700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d673cfc0 0x5641d674f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81db16700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6749a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81eb18700 1 -- 172.21.15.47:0/3181528870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 msgr2=0x5641d6749a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81eb18700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6749a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81eb18700 1 -- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d673cfc0 msgr2=0x5641d674f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81eb18700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d673cfc0 0x5641d674f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81eb18700 1 -- 172.21.15.47:0/3181528870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641d674a7c0 con 0x5641d6742bb0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81e317700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d673cfc0 0x5641d674f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff81eb18700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d6749f80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff8100152b0 tx=0x7ff8100152e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65b2d385e82caab server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff801ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff801ffb700 1 -- 172.21.15.47:0/3181528870 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81001e070 con 0x5641d6742bb0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff801ffb700 1 -- 172.21.15.47:0/3181528870 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7ff810015da0 con 0x5641d6742bb0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff814038410 con 0x5641d6742bb0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff801ffb700 1 -- 172.21.15.47:0/3181528870 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff810018430 con 0x5641d6742bb0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff82ae11b80 10 client.15981 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.360+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5641d67c1950 con 0x5641d6742bb0 2024-05-11T12:35:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff801ffb700 1 -- 172.21.15.47:0/3181528870 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff814038410 con 0x5641d6742bb0 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff801ffb700 1 -- 172.21.15.47:0/3181528870 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7ff8100226e0 con 0x5641d6742bb0 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff82ae11b80 10 client.15981 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff82ae11b80 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff82ae11b80 20 client.15981 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d67c2090 0x5641d67c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff82ae11b80 4 client.15981 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5641d6692440 con 0x5641d67c2090 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.361+0000 7ff81e317700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d67c2090 0x5641d67c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.362+0000 7ff81e317700 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d67c2090 0x5641d67c44b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa1f3d86a9865812 server_cookie=f503609e8d6ed6c6 in_seq=0 out_seq=0 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.362+0000 7ff801ffb700 10 client.15981 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff801ffb700 1 -- 172.21.15.47:0/3181528870 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5641d6692440 con 0x5641d67c2090 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff801ffb700 10 client.15981 handle_command_reply: tid=0 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 client.15981 shutdown 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 msgr2=0x5641d6749f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d6749f80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff8100152b0 tx=0x7ff8100152e0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 shutdown_connections 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5641d67c2090 0x5641d67c44b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5641d6742bb0 0x5641d6749f80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5641d673cfc0 0x5641d674f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 --2- 172.21.15.47:0/3181528870 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5641d6742290 0x5641d6749a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff801ffb700 0 client.15981 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 >> 172.21.15.47:0/3181528870 conn(0x5641d6664bf0 msgr2=0x5641d673fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 shutdown_connections 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 1 -- 172.21.15.47:0/3181528870 wait complete. 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.363+0000 7ff82ae11b80 20 client.15981 trim_cache size 0 max 16384 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8040b2880 msgr2=0x7ff8040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8040b2880 0x7ff8040b4d40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff81819ab50 tx=0x7ff81400b870 comp rx=0 tx=0).stop 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 msgr2=0x7ff81819fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 0x7ff81819fd90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff80800c920 tx=0x7ff80800cc30 comp rx=0 tx=0).stop 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 shutdown_connections 2024-05-11T12:35:50.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff818108900 0x7ff818199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7ff818103800 0x7ff81819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7ff8040b2880 0x7ff8040b4d40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 --2- 172.21.15.47:0/3952680658 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff8181041b0 0x7ff81819fd90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 >> 172.21.15.47:0/3952680658 conn(0x7ff81807a6b0 msgr2=0x7ff818109a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 shutdown_connections 2024-05-11T12:35:50.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:50.364+0000 7ff82057b700 1 -- 172.21.15.47:0/3952680658 wait complete. 2024-05-11T12:35:50.705 DEBUG:teuthology.parallel:result is None 2024-05-11T12:35:50.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-11T12:35:50.779 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-11T12:35:50.779 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-11T12:35:50.791 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-11T12:35:51.656 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-11T12:35:51.910 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:35:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:51 smithi064 ceph-mon[34537]: scrub queued for path: / 2024-05-11T12:35:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:51 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:51 smithi064 ceph-mon[34537]: scrub summary: active paths [/] 2024-05-11T12:35:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:51 smithi064 ceph-mon[34537]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:52.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:51 smithi064 ceph-mon[34537]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 717 MiB/s wr, 1.49k op/s 2024-05-11T12:35:52.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:51 smithi047 ceph-mon[30375]: scrub queued for path: / 2024-05-11T12:35:52.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:51 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:52.185 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:51 smithi047 ceph-mon[30375]: scrub summary: active paths [/] 2024-05-11T12:35:52.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:51 smithi047 ceph-mon[30375]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:52.186 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:51 smithi047 ceph-mon[30375]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 717 MiB/s wr, 1.49k op/s 2024-05-11T12:35:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:51 smithi050 ceph-mon[34072]: scrub queued for path: / 2024-05-11T12:35:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:51 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:51 smithi050 ceph-mon[34072]: scrub summary: active paths [/] 2024-05-11T12:35:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:51 smithi050 ceph-mon[34072]: scrub summary: idle+waiting paths [/] 2024-05-11T12:35:52.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:51 smithi050 ceph-mon[34072]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 717 MiB/s wr, 1.49k op/s 2024-05-11T12:35:54.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:53 smithi047 ceph-mon[30375]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 820 MiB/s wr, 1.68k op/s 2024-05-11T12:35:54.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:53 smithi064 ceph-mon[34537]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 820 MiB/s wr, 1.68k op/s 2024-05-11T12:35:54.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:53 smithi050 ceph-mon[34072]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 820 MiB/s wr, 1.68k op/s 2024-05-11T12:35:54.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.823+0000 7f5a3759e700 1 -- 172.21.15.47:0/3943922092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301076c0 msgr2=0x7f5a3010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.823+0000 7f5a3759e700 1 --2- 172.21.15.47:0/3943922092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301076c0 0x7f5a3010f290 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00b0a0 tx=0x7f5a2c00b3b0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.823+0000 7f5a3759e700 1 -- 172.21.15.47:0/3943922092 shutdown_connections 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.823+0000 7f5a3759e700 1 --2- 172.21.15.47:0/3943922092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301076c0 0x7f5a3010f290 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.823+0000 7f5a3759e700 1 --2- 172.21.15.47:0/3943922092 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a301025e0 0x7f5a301029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.823+0000 7f5a3759e700 1 --2- 172.21.15.47:0/3943922092 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a30102f90 0x7f5a30106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.823+0000 7f5a3759e700 1 -- 172.21.15.47:0/3943922092 >> 172.21.15.47:0/3943922092 conn(0x7f5a300fde70 msgr2=0x7f5a30100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.824+0000 7f5a3759e700 1 -- 172.21.15.47:0/3943922092 shutdown_connections 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.824+0000 7f5a3759e700 1 -- 172.21.15.47:0/3943922092 wait complete. 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 Processor -- start 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 -- start start 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301025e0 0x7f5a3010f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a30102f90 0x7f5a3010fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a301076c0 0x7f5a30110320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301136c0 con 0x7f5a301025e0 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30113540 con 0x7f5a30102f90 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3759e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30113840 con 0x7f5a301076c0 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301025e0 0x7f5a3010f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301025e0 0x7f5a3010f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60528/0 (socket says 172.21.15.47:60528) 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.825+0000 7f5a3659c700 1 -- 172.21.15.47:0/1559311364 learned_addr learned my addr 172.21.15.47:0/1559311364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a30102f90 0x7f5a3010fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a301076c0 0x7f5a30110320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3659c700 1 -- 172.21.15.47:0/1559311364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a301076c0 msgr2=0x7f5a30110320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a301076c0 0x7f5a30110320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3659c700 1 -- 172.21.15.47:0/1559311364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a30102f90 msgr2=0x7f5a3010fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a30102f90 0x7f5a3010fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3659c700 1 -- 172.21.15.47:0/1559311364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a301a7fe0 con 0x7f5a301025e0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a301076c0 0x7f5a30110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301025e0 0x7f5a3010f8a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800b530 tx=0x7f5a2800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a277fe700 1 -- 172.21.15.47:0/1559311364 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28015020 con 0x7f5a301025e0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a277fe700 1 -- 172.21.15.47:0/1559311364 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a28011ce0 con 0x7f5a301025e0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a301a82d0 con 0x7f5a301025e0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a277fe700 1 -- 172.21.15.47:0/1559311364 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28010e40 con 0x7f5a301025e0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.826+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301a8860 con 0x7f5a301025e0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.830+0000 7f5a277fe700 1 -- 172.21.15.47:0/1559311364 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5a28018020 con 0x7f5a301025e0 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.830+0000 7f5a277fe700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5a1c0b2900 0x7f5a1c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.831+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5a1c0b2900 0x7f5a1c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.831+0000 7f5a277fe700 1 -- 172.21.15.47:0/1559311364 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f5a280e1a80 con 0x7f5a301025e0 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.831+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5a1c0b2900 0x7f5a1c0b4dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5a30101530 tx=0x7f5a2000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 Processor -- start 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 -- start start 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560182ade940 con 0x560182b92000 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560182a43da0 con 0x560182b92920 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a43096b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560182ada900 con 0x560182b8cb30 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.832+0000 7f5a3659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a3659c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60534/0 (socket says 172.21.15.47:60534) 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a3659c700 1 -- 172.21.15.47:0/4073670663 learned_addr learned my addr 172.21.15.47:0/4073670663 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a35d9b700 1 -- 172.21.15.47:0/4073670663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 msgr2=0x560182b8cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b8cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a35d9b700 1 -- 172.21.15.47:0/4073670663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 msgr2=0x560182b923e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b923e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a35d9b700 1 -- 172.21.15.47:0/4073670663 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560182a2b8d0 con 0x560182b92920 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b8cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a3659c700 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:35:54.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b8c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a30111330 tx=0x7f5a20012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7245fb58d7daf030 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a24ff9700 1 -- 172.21.15.47:0/4073670663 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20016070 con 0x560182b92920 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a24ff9700 1 -- 172.21.15.47:0/4073670663 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a20014070 con 0x560182b92920 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.833+0000 7f5a24ff9700 1 -- 172.21.15.47:0/4073670663 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20010bf0 con 0x560182b92920 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.834+0000 7f5a43096b80 1 -- 172.21.15.47:0/4073670663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 msgr2=0x560182b8c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.834+0000 7f5a43096b80 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b8c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a30111330 tx=0x7f5a20012000 comp rx=0 tx=0).stop 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.834+0000 7f5a43096b80 1 -- 172.21.15.47:0/4073670663 shutdown_connections 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.834+0000 7f5a43096b80 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b8cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.834+0000 7f5a43096b80 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b923e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.834+0000 7f5a43096b80 1 --2- 172.21.15.47:0/4073670663 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b8c510 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.834+0000 7f5a43096b80 1 -- 172.21.15.47:0/4073670663 >> 172.21.15.47:0/4073670663 conn(0x560182ab8b70 msgr2=0x560182b87090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 -- 172.21.15.47:0/4073670663 shutdown_connections 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 -- 172.21.15.47:0/4073670663 wait complete. 2024-05-11T12:35:54.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 -- start start 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 Processor -- start 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b95a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b95fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x560182b99440 con 0x560182b92000 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x560182a97ed0 con 0x560182b92920 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x560182b995c0 con 0x560182b8cb30 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.835+0000 7f5a43096b80 10 client.0 _ll_register_callbacks cb 0x560182a76950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b95a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b95a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60538/0 (socket says 172.21.15.47:60538) 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 -- 172.21.15.47:0/1739780311 learned_addr learned my addr 172.21.15.47:0/1739780311 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:54.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b95fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 -- 172.21.15.47:0/1739780311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 msgr2=0x560182b986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 -- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 msgr2=0x560182b95fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b95fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 -- 172.21.15.47:0/1739780311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560182a95760 con 0x560182b92000 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b95fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b95a70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5a200106b0 tx=0x7f5a200108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab29b46dd17fda0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a19ffb700 1 -- 172.21.15.47:0/1739780311 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20034060 con 0x560182b92000 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a19ffb700 1 -- 172.21.15.47:0/1739780311 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f5a20010da0 con 0x560182b92000 2024-05-11T12:35:54.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.836+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a20014070 con 0x560182b92000 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a19ffb700 1 -- 172.21.15.47:0/1739780311 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5a20003680 con 0x560182b92000 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 10 client.15993 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560182ba3cf0 con 0x560182b92000 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a19ffb700 1 -- 172.21.15.47:0/1739780311 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a20014070 con 0x560182b92000 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a19ffb700 1 -- 172.21.15.47:0/1739780311 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f5a2001d980 con 0x560182b92000 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 10 client.15993 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 20 client.15993 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182ba6270 0x560182ba8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 4 client.15993 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:35:54.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560182aec320 con 0x560182ba6270 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.837+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182ba6270 0x560182ba8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.838+0000 7f5a19ffb700 10 client.15993 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.838+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182ba6270 0x560182ba8660 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f816668e2cf8614e server_cookie=4329392e022e5cc9 in_seq=0 out_seq=0 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.838+0000 7f5a19ffb700 1 -- 172.21.15.47:0/1739780311 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560182aec320 con 0x560182ba6270 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.838+0000 7f5a19ffb700 10 client.15993 handle_command_reply: tid=0 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.838+0000 7f5a43096b80 1 client.15993 shutdown 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 msgr2=0x560182b95a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b95a70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5a200106b0 tx=0x7f5a200108b0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 shutdown_connections 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182ba6270 0x560182ba8660 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92920 0x560182b95fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a19ffb700 0 client.15993 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1739780311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182b95a70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 >> 172.21.15.47:0/1739780311 conn(0x560182ab8b70 msgr2=0x560182b86660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 shutdown_connections 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 1 -- 172.21.15.47:0/1739780311 wait complete. 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.839+0000 7f5a43096b80 20 client.15993 trim_cache size 0 max 16384 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 Processor -- start 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 -- start start 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182ab8b70 0x560182b8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b93630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182ba24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301ae010 con 0x560182b92000 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b0890 con 0x560182ab8b70 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a43096b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301ae680 con 0x560182b8cb30 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a36d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182ba24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a36d9d700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182ba24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60552/0 (socket says 172.21.15.47:60552) 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a36d9d700 1 -- 172.21.15.47:0/1853827298 learned_addr learned my addr 172.21.15.47:0/1853827298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182ab8b70 0x560182b8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.868+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b93630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a36d9d700 1 -- 172.21.15.47:0/1853827298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 msgr2=0x560182b93630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b93630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a36d9d700 1 -- 172.21.15.47:0/1853827298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182ab8b70 msgr2=0x560182b8c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182ab8b70 0x560182b8c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a36d9d700 1 -- 172.21.15.47:0/1853827298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560182b93fe0 con 0x560182b92000 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a3659c700 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182ab8b70 0x560182b8c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182ba24d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c004c90 tx=0x7f5a2c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c931741c76d471b server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a1bfff700 1 -- 172.21.15.47:0/1853827298 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c0076e0 con 0x560182b92000 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a1bfff700 1 -- 172.21.15.47:0/1853827298 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c007d30 con 0x560182b92000 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.869+0000 7f5a1bfff700 1 -- 172.21.15.47:0/1853827298 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c005770 con 0x560182b92000 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 -- 172.21.15.47:0/1853827298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 msgr2=0x560182ba24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182ba24d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c004c90 tx=0x7f5a2c004cc0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 -- 172.21.15.47:0/1853827298 shutdown_connections 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b92000 0x560182ba24d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182ab8b70 0x560182b8c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 --2- 172.21.15.47:0/1853827298 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182b8cb30 0x560182b93630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 -- 172.21.15.47:0/1853827298 >> 172.21.15.47:0/1853827298 conn(0x560182b8d270 msgr2=0x560182b90bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 -- 172.21.15.47:0/1853827298 shutdown_connections 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 -- 172.21.15.47:0/1853827298 wait complete. 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 -- start start 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.870+0000 7f5a43096b80 1 Processor -- start 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182ab8b70 0x560182bad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b8cb30 0x560182badb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a43096b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92000 0x560182bae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a43096b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301aed50 con 0x560182b8cb30 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a43096b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301b01c0 con 0x560182b92000 2024-05-11T12:35:54.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a43096b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301af420 con 0x560182ab8b70 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a43096b80 10 client.0 _ll_register_callbacks cb 0x560182a76950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b8cb30 0x560182badb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b8cb30 0x560182badb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60562/0 (socket says 172.21.15.47:60562) 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 -- 172.21.15.47:0/3727999098 learned_addr learned my addr 172.21.15.47:0/3727999098 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a3659c700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182ab8b70 0x560182bad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92000 0x560182bae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 -- 172.21.15.47:0/3727999098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182ab8b70 msgr2=0x560182bad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182ab8b70 0x560182bad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 -- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92000 msgr2=0x560182bae050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92000 0x560182bae050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 -- 172.21.15.47:0/3727999098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560182c18200 con 0x560182b8cb30 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a3659c700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182ab8b70 0x560182bad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a36d9d700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92000 0x560182bae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a35d9b700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b8cb30 0x560182badb10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5a20000c00 tx=0x7f5a20005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1ee00d0b947073 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a19ffb700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a19ffb700 1 -- 172.21.15.47:0/3727999098 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20034050 con 0x560182b8cb30 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.871+0000 7f5a19ffb700 1 -- 172.21.15.47:0/3727999098 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f5a20010da0 con 0x560182b8cb30 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2c007d30 con 0x560182b8cb30 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a19ffb700 1 -- 172.21.15.47:0/3727999098 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5a20016040 con 0x560182b8cb30 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 10 client.15999 fetch_fsmap learned FSMap version 20 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560182c19490 con 0x560182b8cb30 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a19ffb700 1 -- 172.21.15.47:0/3727999098 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c007d30 con 0x560182b8cb30 2024-05-11T12:35:54.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a19ffb700 1 -- 172.21.15.47:0/3727999098 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7f5a20025720 con 0x560182b8cb30 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 20 client.15999 populate_metadata read hostname 'smithi047' 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182b92920 0x560182c1ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 4 client.15999 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560182ae63c0 con 0x560182b92920 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.872+0000 7f5a3659c700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182b92920 0x560182c1ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a19ffb700 10 client.15999 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a3659c700 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182b92920 0x560182c1ba30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a3f33548d902bd server_cookie=f46d042df0aa8c27 in_seq=0 out_seq=0 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a19ffb700 1 -- 172.21.15.47:0/3727999098 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560182ae63c0 con 0x560182b92920 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a19ffb700 10 client.15999 handle_command_reply: tid=0 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 client.15999 shutdown 2024-05-11T12:35:54.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b8cb30 msgr2=0x560182badb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b8cb30 0x560182badb10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f5a20000c00 tx=0x7f5a20005600 comp rx=0 tx=0).stop 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 shutdown_connections 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x560182b92920 0x560182c1ba30 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x560182b92000 0x560182bae050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x560182ab8b70 0x560182bad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 --2- 172.21.15.47:0/3727999098 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x560182b8cb30 0x560182badb10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 >> 172.21.15.47:0/3727999098 conn(0x560182b8d270 msgr2=0x560182b94ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a19ffb700 0 client.15999 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 shutdown_connections 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 1 -- 172.21.15.47:0/3727999098 wait complete. 2024-05-11T12:35:54.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.873+0000 7f5a43096b80 20 client.15999 trim_cache size 0 max 16384 2024-05-11T12:35:54.894 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:35:54.894 INFO:teuthology.orchestra.run.smithi047.stdout: "status": "no active scrubs running", 2024-05-11T12:35:54.894 INFO:teuthology.orchestra.run.smithi047.stdout: "scrubs": {} 2024-05-11T12:35:54.894 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5a1c0b2900 msgr2=0x7f5a1c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5a1c0b2900 0x7f5a1c0b4dc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5a30101530 tx=0x7f5a2000a380 comp rx=0 tx=0).stop 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301025e0 msgr2=0x7f5a3010f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301025e0 0x7f5a3010f8a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5a2800b530 tx=0x7f5a2800b840 comp rx=0 tx=0).stop 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 shutdown_connections 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5a301076c0 0x7f5a30110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5a301025e0 0x7f5a3010f8a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5a1c0b2900 0x7f5a1c0b4dc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 --2- 172.21.15.47:0/1559311364 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5a30102f90 0x7f5a3010fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 >> 172.21.15.47:0/1559311364 conn(0x7f5a300fde70 msgr2=0x7f5a300fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 shutdown_connections 2024-05-11T12:35:54.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:54.874+0000 7f5a3759e700 1 -- 172.21.15.47:0/1559311364 wait complete. 2024-05-11T12:35:55.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:54 smithi064 ceph-mon[34537]: scrub summary: idle 2024-05-11T12:35:55.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:54 smithi047 ceph-mon[30375]: scrub summary: idle 2024-05-11T12:35:55.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:54 smithi050 ceph-mon[34072]: scrub summary: idle 2024-05-11T12:35:55.595 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-11T12:35:55.595 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:35:56.008 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:35:56.114 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:55 smithi064 ceph-mon[34537]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 622 MiB/s wr, 1.21k op/s 2024-05-11T12:35:56.143 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:55 smithi047 ceph-mon[30375]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 622 MiB/s wr, 1.21k op/s 2024-05-11T12:35:56.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:55 smithi050 ceph-mon[34072]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 622 MiB/s wr, 1.21k op/s 2024-05-11T12:35:58.113 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:57 smithi047 ceph-mon[30375]: pgmap v740: 97 pgs: 97 active+clean; 9.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 579 MiB/s wr, 1.34k op/s 2024-05-11T12:35:58.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:57 smithi050 ceph-mon[34072]: pgmap v740: 97 pgs: 97 active+clean; 9.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 579 MiB/s wr, 1.34k op/s 2024-05-11T12:35:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.219+0000 7f5bac7e0700 1 -- 172.21.15.47:0/414302762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4109730 msgr2=0x7f5ba4109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.219+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/414302762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4109730 0x7f5ba4109b10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009990 tx=0x7f5ba0009ca0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 -- 172.21.15.47:0/414302762 shutdown_connections 2024-05-11T12:35:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/414302762 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ba4102940 0x7f5ba4110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/414302762 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4109730 0x7f5ba4109b10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/414302762 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4101f80 0x7f5ba4102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 -- 172.21.15.47:0/414302762 >> 172.21.15.47:0/414302762 conn(0x7f5ba40776c0 msgr2=0x7f5ba4077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 -- 172.21.15.47:0/414302762 shutdown_connections 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 -- 172.21.15.47:0/414302762 wait complete. 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 Processor -- start 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.220+0000 7f5bac7e0700 1 -- start start 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5bac7e0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4101f80 0x7f5ba419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5bac7e0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ba4102940 0x7f5ba419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5bac7e0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4109730 0x7f5ba41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5bac7e0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba4113760 con 0x7f5ba4101f80 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5bac7e0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41135e0 con 0x7f5ba4109730 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5bac7e0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba41138e0 con 0x7f5ba4102940 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4101f80 0x7f5ba419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:58.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4101f80 0x7f5ba419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46514/0 (socket says 172.21.15.47:46514) 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 -- 172.21.15.47:0/3555467620 learned_addr learned my addr 172.21.15.47:0/3555467620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5ba9d7b700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ba4102940 0x7f5ba419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baad7d700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4109730 0x7f5ba41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 -- 172.21.15.47:0/3555467620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ba4102940 msgr2=0x7f5ba419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ba4102940 0x7f5ba419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 -- 172.21.15.47:0/3555467620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4109730 msgr2=0x7f5ba41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4109730 0x7f5ba41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 -- 172.21.15.47:0/3555467620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba419a100 con 0x7f5ba4101f80 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5ba9d7b700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ba4102940 0x7f5ba419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5baa57c700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4101f80 0x7f5ba419f7e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00057f0 tx=0x7f5ba0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.221+0000 7f5b9b7fe700 1 -- 172.21.15.47:0/3555467620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0027070 con 0x7f5ba4101f80 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.222+0000 7f5b9b7fe700 1 -- 172.21.15.47:0/3555467620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ba00041c0 con 0x7f5ba4101f80 2024-05-11T12:35:58.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.222+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba419a390 con 0x7f5ba4101f80 2024-05-11T12:35:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.222+0000 7f5b9b7fe700 1 -- 172.21.15.47:0/3555467620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0022e60 con 0x7f5ba4101f80 2024-05-11T12:35:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.222+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba41a66b0 con 0x7f5ba4101f80 2024-05-11T12:35:58.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.223+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba406a240 con 0x7f5ba4101f80 2024-05-11T12:35:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.224+0000 7f5b9b7fe700 1 -- 172.21.15.47:0/3555467620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ba002a070 con 0x7f5ba4101f80 2024-05-11T12:35:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.225+0000 7f5b9b7fe700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5b900b2900 0x7f5b900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:35:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.225+0000 7f5ba9d7b700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5b900b2900 0x7f5b900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:35:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.225+0000 7f5b9b7fe700 1 -- 172.21.15.47:0/3555467620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f5ba00e3570 con 0x7f5ba4101f80 2024-05-11T12:35:58.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.225+0000 7f5ba9d7b700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5b900b2900 0x7f5b900b4dc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4077c50 tx=0x7f5b94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:35:58.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.228+0000 7f5b9b7fe700 1 -- 172.21.15.47:0/3555467620 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5ba0013070 con 0x7f5ba4101f80 2024-05-11T12:35:58.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:57 smithi064 ceph-mon[34537]: pgmap v740: 97 pgs: 97 active+clean; 9.7 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 579 MiB/s wr, 1.34k op/s 2024-05-11T12:35:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.489+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ba402e570 con 0x7f5ba4101f80 2024-05-11T12:35:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.490+0000 7f5b9b7fe700 1 -- 172.21.15.47:0/3555467620 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f5ba00ab250 con 0x7f5ba4101f80 2024-05-11T12:35:58.491 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:35:58.491 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:35:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5b900b2900 msgr2=0x7f5b900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5b900b2900 0x7f5b900b4dc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4077c50 tx=0x7f5b94008040 comp rx=0 tx=0).stop 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4101f80 msgr2=0x7f5ba419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4101f80 0x7f5ba419f7e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ba00057f0 tx=0x7f5ba0003730 comp rx=0 tx=0).stop 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 shutdown_connections 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f5b900b2900 0x7f5b900b4dc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5ba4109730 0x7f5ba41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ba4101f80 0x7f5ba419f7e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 --2- 172.21.15.47:0/3555467620 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5ba4102940 0x7f5ba419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 >> 172.21.15.47:0/3555467620 conn(0x7f5ba40776c0 msgr2=0x7f5ba4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 shutdown_connections 2024-05-11T12:35:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:35:58.492+0000 7f5bac7e0700 1 -- 172.21.15.47:0/3555467620 wait complete. 2024-05-11T12:35:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:35:59.119 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:58 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3555467620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:35:59.154 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-11T12:35:59.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:58 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3555467620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:35:59.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:58 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3555467620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:35:59.418 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:00.166 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:35:59 smithi047 ceph-mon[30375]: pgmap v741: 97 pgs: 97 active+clean; 8.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 509 MiB/s wr, 1.45k op/s 2024-05-11T12:36:00.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:35:59 smithi050 ceph-mon[34072]: pgmap v741: 97 pgs: 97 active+clean; 8.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 509 MiB/s wr, 1.45k op/s 2024-05-11T12:36:00.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:35:59 smithi064 ceph-mon[34537]: pgmap v741: 97 pgs: 97 active+clean; 8.5 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 509 MiB/s wr, 1.45k op/s 2024-05-11T12:36:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.768+0000 7fd3fc1af700 1 -- 172.21.15.47:0/1439193866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 msgr2=0x7fd3f410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.768+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/1439193866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f410f310 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00b0d0 tx=0x7fd3ec00b3e0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 -- 172.21.15.47:0/1439193866 shutdown_connections 2024-05-11T12:36:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/1439193866 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f410f310 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/1439193866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3f4108540 0x7fd3f4108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/1439193866 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f4102490 0x7fd3f4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 -- 172.21.15.47:0/1439193866 >> 172.21.15.47:0/1439193866 conn(0x7fd3f40794f0 msgr2=0x7fd3f4079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 -- 172.21.15.47:0/1439193866 shutdown_connections 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 -- 172.21.15.47:0/1439193866 wait complete. 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 Processor -- start 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 -- start start 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f4102490 0x7fd3f4104100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f4104640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.769+0000 7fd3fc1af700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3f4108540 0x7fd3f4104b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3fc1af700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41126a0 con 0x7fd3f4102e50 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3fc1af700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4112520 con 0x7fd3f4108540 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3fc1af700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f4112820 con 0x7fd3f4102490 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f4104640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f4104640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46536/0 (socket says 172.21.15.47:46536) 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 -- 172.21.15.47:0/2666253347 learned_addr learned my addr 172.21.15.47:0/2666253347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f4102490 0x7fd3f4104100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 -- 172.21.15.47:0/2666253347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f4102490 msgr2=0x7fd3f4104100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f4102490 0x7fd3f4104100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 -- 172.21.15.47:0/2666253347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3f4108540 msgr2=0x7fd3f4104b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3f4108540 0x7fd3f4104b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3f4108540 0x7fd3f4104b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 -- 172.21.15.47:0/2666253347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41a80c0 con 0x7fd3f4102e50 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3f4108540 0x7fd3f4104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f4104640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400d980 tx=0x7fd3e400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3eaffd700 1 -- 172.21.15.47:0/2666253347 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4017070 con 0x7fd3f4102e50 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41a83b0 con 0x7fd3f4102e50 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3eaffd700 1 -- 172.21.15.47:0/2666253347 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3e4004830 con 0x7fd3f4102e50 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3eaffd700 1 -- 172.21.15.47:0/2666253347 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4005440 con 0x7fd3f4102e50 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.770+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a88a0 con 0x7fd3f4102e50 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.773+0000 7fd3eaffd700 1 -- 172.21.15.47:0/2666253347 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd3e40055e0 con 0x7fd3f4102e50 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.773+0000 7fd3eaffd700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3e00b2880 0x7fd3e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3e00b2880 0x7fd3e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd3eaffd700 1 -- 172.21.15.47:0/2666253347 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd3e40e2290 con 0x7fd3f4102e50 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 Processor -- start 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3e00b2880 0x7fd3e00b4d40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0005950 tx=0x7fd3f00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 -- start start 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bb0a0 0x5561bb1bb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bba60 0x5561bb1c7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5561bb10d860 con 0x5561bb1bba60 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561bb0ac7c0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.774+0000 7fd406a45b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561bb109820 con 0x5561bb1c0ad0 2024-05-11T12:36:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bba60 0x5561bb1c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bba60 0x5561bb1c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46552/0 (socket says 172.21.15.47:46552) 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 -- 172.21.15.47:0/1933661682 learned_addr learned my addr 172.21.15.47:0/1933661682 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3f974a700 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bb0a0 0x5561bb1bb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 -- 172.21.15.47:0/1933661682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 msgr2=0x5561bb1c0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 -- 172.21.15.47:0/1933661682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bb0a0 msgr2=0x5561bb1bb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bb0a0 0x5561bb1bb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 -- 172.21.15.47:0/1933661682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561bb05a8d0 con 0x5561bb1bba60 2024-05-11T12:36:01.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bba60 0x5561bb1c7890 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d000 tx=0x7fd3ec009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08a83fa29967aaa server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3dffff700 1 -- 172.21.15.47:0/1933661682 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec020070 con 0x5561bb1bba60 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3dffff700 1 -- 172.21.15.47:0/1933661682 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3ec0092d0 con 0x5561bb1bba60 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.775+0000 7fd3dffff700 1 -- 172.21.15.47:0/1933661682 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec007dc0 con 0x5561bb1bba60 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 -- 172.21.15.47:0/1933661682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bba60 msgr2=0x5561bb1c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bba60 0x5561bb1c7890 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00d000 tx=0x7fd3ec009b80 comp rx=0 tx=0).stop 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 -- 172.21.15.47:0/1933661682 shutdown_connections 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bba60 0x5561bb1c7890 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1933661682 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bb0a0 0x5561bb1bb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 -- 172.21.15.47:0/1933661682 >> 172.21.15.47:0/1933661682 conn(0x5561bb0e7e60 msgr2=0x5561bb1b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 -- 172.21.15.47:0/1933661682 shutdown_connections 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.776+0000 7fd406a45b80 1 -- 172.21.15.47:0/1933661682 wait complete. 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 -- start start 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 Processor -- start 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bba60 0x5561bb1c4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5561bb1c7e30 con 0x5561bb1bb0a0 2024-05-11T12:36:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x5561bb073c30 con 0x5561bb1c0ad0 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x5561bb1c7fb0 con 0x5561bb1bba60 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd406a45b80 10 client.0 _ll_register_callbacks cb 0x5561bb0a5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd3f9f4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd3f9f4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46562/0 (socket says 172.21.15.47:46562) 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/2603405258 learned_addr learned my addr 172.21.15.47:0/2603405258 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.777+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/2603405258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bba60 msgr2=0x5561bb1c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bba60 0x5561bb1c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bba60 0x5561bb1c4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1c0ad0 msgr2=0x5561bb1c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/2603405258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561bb0a9690 con 0x5561bb1bb0a0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bba60 0x5561bb1c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1c7460 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000ff60 tx=0x7fd3f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24576cf9be09d532 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/2603405258 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0028670 con 0x5561bb1bb0a0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/2603405258 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd3f0028810 con 0x5561bb1bb0a0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ec0092d0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/2603405258 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd3f0028a10 con 0x5561bb1bb0a0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd406a45b80 10 client.16017 fetch_fsmap learned FSMap version 20 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5561bb1d26a0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.778+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/2603405258 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3ec0092d0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/2603405258 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd3f0032ba0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd406a45b80 10 client.16017 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd406a45b80 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd406a45b80 20 client.16017 populate_metadata read hostname 'smithi047' 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb1d4c30 0x5561bb1d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd406a45b80 4 client.16017 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561bb11b240 con 0x5561bb1d4c30 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb1d4c30 0x5561bb1d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb1d4c30 0x5561bb1d7040 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2374c01d01c2d4 server_cookie=f8bc2ab4b042c899 in_seq=0 out_seq=0 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.779+0000 7fd3dd7fa700 10 client.16017 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/2603405258 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5561bb11b240 con 0x5561bb1d4c30 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd3dd7fa700 10 client.16017 handle_command_reply: tid=0 2024-05-11T12:36:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 client.16017 shutdown 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 msgr2=0x5561bb1c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1c7460 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000ff60 tx=0x7fd3f0004750 comp rx=0 tx=0).stop 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 shutdown_connections 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb1d4c30 0x5561bb1d7040 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1c0ad0 0x5561bb1c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1c7460 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2603405258 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bba60 0x5561bb1c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd3dd7fa700 0 client.16017 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 >> 172.21.15.47:0/2603405258 conn(0x5561bb0e7e60 msgr2=0x5561bb1b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 shutdown_connections 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.780+0000 7fd406a45b80 1 -- 172.21.15.47:0/2603405258 wait complete. 2024-05-11T12:36:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.781+0000 7fd406a45b80 20 client.16017 trim_cache size 0 max 16384 2024-05-11T12:36:01.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.795+0000 7fd406a45b80 1 Processor -- start 2024-05-11T12:36:01.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd406a45b80 1 -- start start 2024-05-11T12:36:01.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bb0a0 0x5561bb1d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1c0ad0 0x5561bb1bb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd406a45b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ae010 con 0x5561bb1c0ad0 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd406a45b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41b0890 con 0x5561bb1bba60 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd406a45b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41ae680 con 0x5561bb1bb0a0 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1c0ad0 0x5561bb1bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1c0ad0 0x5561bb1bb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46568/0 (socket says 172.21.15.47:46568) 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 -- 172.21.15.47:0/2570235343 learned_addr learned my addr 172.21.15.47:0/2570235343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3f974a700 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bb0a0 0x5561bb1d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 -- 172.21.15.47:0/2570235343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bb0a0 msgr2=0x5561bb1d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bb0a0 0x5561bb1d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 -- 172.21.15.47:0/2570235343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 msgr2=0x5561bb1c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 -- 172.21.15.47:0/2570235343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561bb1c50c0 con 0x5561bb1c0ad0 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.796+0000 7fd3fa74c700 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1c0ad0 0x5561bb1bb480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00b820 tx=0x7fd3ec0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a41bd039abbc6df server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd3df7fe700 1 -- 172.21.15.47:0/2570235343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec020040 con 0x5561bb1c0ad0 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd3df7fe700 1 -- 172.21.15.47:0/2570235343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3ec004c50 con 0x5561bb1c0ad0 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd3df7fe700 1 -- 172.21.15.47:0/2570235343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec025450 con 0x5561bb1c0ad0 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 -- 172.21.15.47:0/2570235343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1c0ad0 msgr2=0x5561bb1bb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1c0ad0 0x5561bb1bb480 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00b820 tx=0x7fd3ec0092e0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 -- 172.21.15.47:0/2570235343 shutdown_connections 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1c0ad0 0x5561bb1bb480 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1bb0a0 0x5561bb1d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 --2- 172.21.15.47:0/2570235343 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 -- 172.21.15.47:0/2570235343 >> 172.21.15.47:0/2570235343 conn(0x5561bb0e7e60 msgr2=0x5561bb1ba090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 -- 172.21.15.47:0/2570235343 shutdown_connections 2024-05-11T12:36:01.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.797+0000 7fd406a45b80 1 -- 172.21.15.47:0/2570235343 wait complete. 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 -- start start 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 Processor -- start 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 0x5561bb1dc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41aed50 con 0x5561bb1bb0a0 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41b01c0 con 0x5561bb1bba60 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f41af420 con 0x5561bb1c0ad0 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd406a45b80 10 client.0 _ll_register_callbacks cb 0x5561bb0a5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd3f9f4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.798+0000 7fd3f9f4b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46572/0 (socket says 172.21.15.47:46572) 2024-05-11T12:36:01.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/1951039400 learned_addr learned my addr 172.21.15.47:0/1951039400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f974a700 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/1951039400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 msgr2=0x5561bb1dc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 0x5561bb1dc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 msgr2=0x5561bb1dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f9f4b700 1 -- 172.21.15.47:0/1951039400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561bb1dccb0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f974a700 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3f9f4b700 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1cd860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f000ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ee86bb07e80be8 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3dd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/1951039400 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f003f060 con 0x5561bb1bb0a0 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/1951039400 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fd3f0028470 con 0x5561bb1bb0a0 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3ec004c50 con 0x5561bb1bb0a0 2024-05-11T12:36:01.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/1951039400 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd3f0011ab0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd406a45b80 10 client.16023 fetch_fsmap learned FSMap version 20 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.799+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5561bb23f570 con 0x5561bb1bb0a0 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/1951039400 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3ec004c50 con 0x5561bb1bb0a0 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/1951039400 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 20) ==== 8322+0+0 (secure 0 0 0) 0x7fd3f00047c0 con 0x5561bb1bb0a0 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd406a45b80 10 client.16023 fetch_fsmap finished waiting for FSMap version 20 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd406a45b80 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.h 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd406a45b80 20 client.16023 populate_metadata read hostname 'smithi047' 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb23fcb0 0x5561bb242110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd406a45b80 4 client.16023 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 --> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561bb1152e0 con 0x5561bb23fcb0 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd3f974a700 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb23fcb0 0x5561bb242110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd3dd7fa700 10 client.16023 ms_handle_connect on v2:172.21.15.50:6836/3384616382 2024-05-11T12:36:01.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.800+0000 7fd3f974a700 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb23fcb0 0x5561bb242110 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=804b7948b4e37799 server_cookie=3b791309a5be5f32 in_seq=0 out_seq=0 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd3dd7fa700 1 -- 172.21.15.47:0/1951039400 <== mds.0 v2:172.21.15.50:6836/3384616382 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561bb1152e0 con 0x5561bb23fcb0 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd3dd7fa700 10 client.16023 handle_command_reply: tid=0 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 client.16023 shutdown 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 msgr2=0x5561bb1cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1cd860 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0000c00 tx=0x7fd3f000ba60 comp rx=0 tx=0).stop 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 shutdown_connections 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:6836/3384616382,v1:172.21.15.50:6837/3384616382] conn(0x5561bb23fcb0 0x5561bb242110 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x5561bb1c0ad0 0x5561bb1dc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5561bb1bb0a0 0x5561bb1cd860 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 --2- 172.21.15.47:0/1951039400 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x5561bb1bba60 0x5561bb1dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd3dd7fa700 0 client.16023 ms_handle_reset on v2:172.21.15.50:6836/3384616382 2024-05-11T12:36:01.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 >> 172.21.15.47:0/1951039400 conn(0x5561bb0e7e60 msgr2=0x5561bb1ba920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:01.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 shutdown_connections 2024-05-11T12:36:01.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 1 -- 172.21.15.47:0/1951039400 wait complete. 2024-05-11T12:36:01.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.801+0000 7fd406a45b80 20 client.16023 trim_cache size 0 max 16384 2024-05-11T12:36:01.809 INFO:teuthology.orchestra.run.smithi047.stdout:[] 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3e00b2880 msgr2=0x7fd3e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3e00b2880 0x7fd3e00b4d40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0005950 tx=0x7fd3f00058e0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 msgr2=0x7fd3f4104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f4104640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400d980 tx=0x7fd3e400dd40 comp rx=0 tx=0).stop 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 shutdown_connections 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fd3e00b2880 0x7fd3e00b4d40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3f4108540 0x7fd3f4104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fd3f4102490 0x7fd3f4104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 --2- 172.21.15.47:0/2666253347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd3f4102e50 0x7fd3f4104640 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:01.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 >> 172.21.15.47:0/2666253347 conn(0x7fd3f40794f0 msgr2=0x7fd3f41016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:01.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 shutdown_connections 2024-05-11T12:36:01.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:01.802+0000 7fd3fc1af700 1 -- 172.21.15.47:0/2666253347 wait complete. 2024-05-11T12:36:02.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:01 smithi047 ceph-mon[30375]: pgmap v742: 97 pgs: 97 active+clean; 8.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 291 MiB/s wr, 1.06k op/s 2024-05-11T12:36:02.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:01 smithi050 ceph-mon[34072]: pgmap v742: 97 pgs: 97 active+clean; 8.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 291 MiB/s wr, 1.06k op/s 2024-05-11T12:36:02.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:01 smithi064 ceph-mon[34537]: pgmap v742: 97 pgs: 97 active+clean; 8.2 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 291 MiB/s wr, 1.06k op/s 2024-05-11T12:36:02.614 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_308506802b0bdea4df7050cfc1fcbdc04fea6a6e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-11T12:36:02.615 INFO:tasks.fwd_scrub:done joining 2024-05-11T12:36:02.615 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-11T12:36:02.626 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 -- 172.21.15.47:0/468015796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 msgr2=0x7fe00c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 --2- 172.21.15.47:0/468015796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c10abe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe004009ca0 tx=0x7fe00402a8c0 comp rx=0 tx=0).stop 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 -- 172.21.15.47:0/468015796 shutdown_connections 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 --2- 172.21.15.47:0/468015796 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c10abe0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 --2- 172.21.15.47:0/468015796 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c105e60 0x7fe00c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 --2- 172.21.15.47:0/468015796 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe00c103530 0x7fe00c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 -- 172.21.15.47:0/468015796 >> 172.21.15.47:0/468015796 conn(0x7fe00c0fcf20 msgr2=0x7fe00c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:02.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 -- 172.21.15.47:0/468015796 shutdown_connections 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 -- 172.21.15.47:0/468015796 wait complete. 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 Processor -- start 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.858+0000 7fe0108e1640 1 -- start start 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe0108e1640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c103530 0x7fe00c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe0108e1640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe00c105e60 0x7fe00c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe0108e1640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe0108e1640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10e5e0 con 0x7fe00c108790 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe0108e1640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10e460 con 0x7fe00c105e60 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe0108e1640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c10e760 con 0x7fe00c103530 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c1a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46580/0 (socket says 172.21.15.47:46580) 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00a59c640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c103530 0x7fe00c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:02.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 -- 172.21.15.47:0/1659772342 learned_addr learned my addr 172.21.15.47:0/1659772342 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe003fff640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe00c105e60 0x7fe00c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 -- 172.21.15.47:0/1659772342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c103530 msgr2=0x7fe00c198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c103530 0x7fe00c198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 -- 172.21.15.47:0/1659772342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe00c105e60 msgr2=0x7fe00c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe00c105e60 0x7fe00c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 -- 172.21.15.47:0/1659772342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c1a07b0 con 0x7fe00c108790 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.859+0000 7fe00ad9d640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c1a0140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe0040097b0 tx=0x7fe004003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.860+0000 7fe0037fe640 1 -- 172.21.15.47:0/1659772342 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004004240 con 0x7fe00c108790 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.860+0000 7fe0037fe640 1 -- 172.21.15.47:0/1659772342 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0040317c0 con 0x7fe00c108790 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.860+0000 7fe0037fe640 1 -- 172.21.15.47:0/1659772342 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00402fc70 con 0x7fe00c108790 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.860+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c1a0a40 con 0x7fe00c108790 2024-05-11T12:36:02.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.860+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c1a0f90 con 0x7fe00c108790 2024-05-11T12:36:02.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.861+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00c0fea10 con 0x7fe00c108790 2024-05-11T12:36:02.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.862+0000 7fe0037fe640 1 -- 172.21.15.47:0/1659772342 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe0040043e0 con 0x7fe00c108790 2024-05-11T12:36:02.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.863+0000 7fe0037fe640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff80b27f0 0x7fdff80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:02.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.863+0000 7fe00a59c640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff80b27f0 0x7fdff80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:02.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.863+0000 7fe0037fe640 1 -- 172.21.15.47:0/1659772342 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fe004103c90 con 0x7fe00c108790 2024-05-11T12:36:02.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.864+0000 7fe00a59c640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff80b27f0 0x7fdff80b4cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005d00 tx=0x7fdffc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:02.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:02.867+0000 7fe0037fe640 1 -- 172.21.15.47:0/1659772342 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe0040cb880 con 0x7fe00c108790 2024-05-11T12:36:03.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.060+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe00c02dd30 con 0x7fe00c108790 2024-05-11T12:36:03.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.062+0000 7fe0037fe640 1 -- 172.21.15.47:0/1659772342 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22917 (secure 0 0 0) 0x7fe0040098b0 con 0x7fe00c108790 2024-05-11T12:36:03.063 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:36:03.063 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:22:43.640272+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:36:03.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.063+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff80b27f0 msgr2=0x7fdff80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.063+0000 7fe0108e1640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff80b27f0 0x7fdff80b4cb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005d00 tx=0x7fdffc00b410 comp rx=0 tx=0).stop 2024-05-11T12:36:03.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 msgr2=0x7fe00c1a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c1a0140 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe0040097b0 tx=0x7fe004003c20 comp rx=0 tx=0).stop 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 shutdown_connections 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fdff80b27f0 0x7fdff80b4cb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe00c108790 0x7fe00c1a0140 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe00c105e60 0x7fe00c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 --2- 172.21.15.47:0/1659772342 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fe00c103530 0x7fe00c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 >> 172.21.15.47:0/1659772342 conn(0x7fe00c0fcf20 msgr2=0x7fe00c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 shutdown_connections 2024-05-11T12:36:03.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.064+0000 7fe0108e1640 1 -- 172.21.15.47:0/1659772342 wait complete. 2024-05-11T12:36:03.077 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-11T12:16:11.517702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-11T12:21:06.576652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-11T12:21:10.443533+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-11T12:36:03.077 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-11T12:36:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 -- 172.21.15.47:0/1455726581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474102b90 msgr2=0x7f5474103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 --2- 172.21.15.47:0/1455726581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474102b90 0x7f5474103010 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5464009900 tx=0x7f546402a410 comp rx=0 tx=0).stop 2024-05-11T12:36:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 -- 172.21.15.47:0/1455726581 shutdown_connections 2024-05-11T12:36:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 --2- 172.21.15.47:0/1455726581 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5474103550 0x7f5474109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 --2- 172.21.15.47:0/1455726581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474102b90 0x7f5474103010 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 --2- 172.21.15.47:0/1455726581 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54741019a0 0x7f5474101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 -- 172.21.15.47:0/1455726581 >> 172.21.15.47:0/1455726581 conn(0x7f54740fcf30 msgr2=0x7f54740ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 -- 172.21.15.47:0/1455726581 shutdown_connections 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 -- 172.21.15.47:0/1455726581 wait complete. 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 Processor -- start 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.185+0000 7f547c62a640 1 -- start start 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54741019a0 0x7f547406a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5474102b90 0x7f547419da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474103550 0x7f54741a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f547410d4b0 con 0x7f5474103550 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f547410d330 con 0x7f54741019a0 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f547410d630 con 0x7f5474102b90 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474103550 0x7f54741a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474103550 0x7f54741a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46598/0 (socket says 172.21.15.47:46598) 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 -- 172.21.15.47:0/271426420 learned_addr learned my addr 172.21.15.47:0/271426420 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f5479bc5640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5474102b90 0x7f547419da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547a3c6640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54741019a0 0x7f547406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 -- 172.21.15.47:0/271426420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5474102b90 msgr2=0x7f547419da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5474102b90 0x7f547419da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 -- 172.21.15.47:0/271426420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54741019a0 msgr2=0x7f547406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54741019a0 0x7f547406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 -- 172.21.15.47:0/271426420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54741a0940 con 0x7f5474103550 2024-05-11T12:36:03.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f5479bc5640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5474102b90 0x7f547419da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:36:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547abc7640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474103550 0x7f54741a0080 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f546c00cad0 tx=0x7f546c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f546b7fe640 1 -- 172.21.15.47:0/271426420 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c00ae20 con 0x7f5474103550 2024-05-11T12:36:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f546b7fe640 1 -- 172.21.15.47:0/271426420 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546c005020 con 0x7f5474103550 2024-05-11T12:36:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54741a0c30 con 0x7f5474103550 2024-05-11T12:36:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f546b7fe640 1 -- 172.21.15.47:0/271426420 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c00f6b0 con 0x7f5474103550 2024-05-11T12:36:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.186+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a1130 con 0x7f5474103550 2024-05-11T12:36:03.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.187+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5474108e80 con 0x7f5474103550 2024-05-11T12:36:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.189+0000 7f546b7fe640 1 -- 172.21.15.47:0/271426420 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f546c0051d0 con 0x7f5474103550 2024-05-11T12:36:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.190+0000 7f546b7fe640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f54600b27f0 0x7f54600b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.190+0000 7f547a3c6640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f54600b27f0 0x7f54600b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.190+0000 7f546b7fe640 1 -- 172.21.15.47:0/271426420 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f546c0e2720 con 0x7f5474103550 2024-05-11T12:36:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.190+0000 7f547a3c6640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f54600b27f0 0x7f54600b4cb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5470005d00 tx=0x7f5470005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:03.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.193+0000 7f546b7fe640 1 -- 172.21.15.47:0/271426420 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f546c014030 con 0x7f5474103550 2024-05-11T12:36:03.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.383+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f547402dd40 con 0x7f5474103550 2024-05-11T12:36:03.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.383+0000 7f546b7fe640 1 -- 172.21.15.47:0/271426420 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f546c0aa370 con 0x7f5474103550 2024-05-11T12:36:03.385 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-05-11T12:36:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f54600b27f0 msgr2=0x7f54600b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f54600b27f0 0x7f54600b4cb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5470005d00 tx=0x7f5470005c50 comp rx=0 tx=0).stop 2024-05-11T12:36:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474103550 msgr2=0x7f54741a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474103550 0x7f54741a0080 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f546c00cad0 tx=0x7f546c00cf90 comp rx=0 tx=0).stop 2024-05-11T12:36:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 shutdown_connections 2024-05-11T12:36:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f54600b27f0 0x7f54600b4cb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5474103550 0x7f54741a0080 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f5474102b90 0x7f547419da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 --2- 172.21.15.47:0/271426420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f54741019a0 0x7f547406a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 >> 172.21.15.47:0/271426420 conn(0x7f54740fcf30 msgr2=0x7f5474106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 shutdown_connections 2024-05-11T12:36:03.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.385+0000 7f547c62a640 1 -- 172.21.15.47:0/271426420 wait complete. 2024-05-11T12:36:03.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-11T12:36:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3616069830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e24102ba0 msgr2=0x7f1e24103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3616069830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e24102ba0 0x7f1e24103020 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009900 tx=0x7f1e0c02a410 comp rx=0 tx=0).stop 2024-05-11T12:36:03.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3616069830 shutdown_connections 2024-05-11T12:36:03.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3616069830 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e24103560 0x7f1e24109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3616069830 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e24102ba0 0x7f1e24103020 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3616069830 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e241019b0 0x7f1e24101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3616069830 >> 172.21.15.47:0/3616069830 conn(0x7f1e240fcf40 msgr2=0x7f1e240ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:03.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3616069830 shutdown_connections 2024-05-11T12:36:03.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.506+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3616069830 wait complete. 2024-05-11T12:36:03.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 Processor -- start 2024-05-11T12:36:03.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 -- start start 2024-05-11T12:36:03.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e241019b0 0x7f1e2406bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e24102ba0 0x7f1e2419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e24103560 0x7f1e241a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410d550 con 0x7f1e241019b0 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410d3d0 con 0x7f1e24102ba0 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e28a6d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2410d6d0 con 0x7f1e24103560 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e2259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e241019b0 0x7f1e2406bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e2259c640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e241019b0 0x7f1e2406bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46624/0 (socket says 172.21.15.47:46624) 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e2259c640 1 -- 172.21.15.47:0/3576055248 learned_addr learned my addr 172.21.15.47:0/3576055248 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e19d9b640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e24102ba0 0x7f1e2419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e22d9d640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e24103560 0x7f1e241a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e2259c640 1 -- 172.21.15.47:0/3576055248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e24103560 msgr2=0x7f1e241a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e2259c640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e24103560 0x7f1e241a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e2259c640 1 -- 172.21.15.47:0/3576055248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e24102ba0 msgr2=0x7f1e2419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.507+0000 7f1e2259c640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e24102ba0 0x7f1e2419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.508+0000 7f1e2259c640 1 -- 172.21.15.47:0/3576055248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e241a0970 con 0x7f1e241019b0 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.508+0000 7f1e2259c640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e241019b0 0x7f1e2406bc60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b450 tx=0x7f1e1400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.508+0000 7f1e1bfff640 1 -- 172.21.15.47:0/3576055248 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e14015020 con 0x7f1e241019b0 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.508+0000 7f1e1bfff640 1 -- 172.21.15.47:0/3576055248 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e14011d40 con 0x7f1e241019b0 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.508+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e241a0c60 con 0x7f1e241019b0 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.508+0000 7f1e1bfff640 1 -- 172.21.15.47:0/3576055248 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1400f340 con 0x7f1e241019b0 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.508+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e241a11d0 con 0x7f1e241019b0 2024-05-11T12:36:03.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.509+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e24108cf0 con 0x7f1e241019b0 2024-05-11T12:36:03.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.510+0000 7f1e1bfff640 1 -- 172.21.15.47:0/3576055248 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1e14018020 con 0x7f1e241019b0 2024-05-11T12:36:03.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.511+0000 7f1e1bfff640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e100b2870 0x7f1e100b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.511+0000 7f1e19d9b640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e100b2870 0x7f1e100b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.511+0000 7f1e1bfff640 1 -- 172.21.15.47:0/3576055248 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1e140e1d50 con 0x7f1e241019b0 2024-05-11T12:36:03.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.512+0000 7f1e19d9b640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e100b2870 0x7f1e100b4d30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c0314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:03.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.516+0000 7f1e1bfff640 1 -- 172.21.15.47:0/3576055248 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1e1401f020 con 0x7f1e241019b0 2024-05-11T12:36:03.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.707+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1e24063660 con 0x7f1e241019b0 2024-05-11T12:36:03.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.707+0000 7f1e1bfff640 1 -- 172.21.15.47:0/3576055248 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7f1e1400d040 con 0x7f1e241019b0 2024-05-11T12:36:03.708 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-05-11T12:36:03.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e100b2870 msgr2=0x7f1e100b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e100b2870 0x7f1e100b4d30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c0314a0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e241019b0 msgr2=0x7f1e2406bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e241019b0 0x7f1e2406bc60 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b450 tx=0x7f1e1400b910 comp rx=0 tx=0).stop 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 shutdown_connections 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1e100b2870 0x7f1e100b4d30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e24103560 0x7f1e241a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e24102ba0 0x7f1e2419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 --2- 172.21.15.47:0/3576055248 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e241019b0 0x7f1e2406bc60 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 >> 172.21.15.47:0/3576055248 conn(0x7f1e240fcf40 msgr2=0x7f1e24105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 shutdown_connections 2024-05-11T12:36:03.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.709+0000 7f1e28a6d640 1 -- 172.21.15.47:0/3576055248 wait complete. 2024-05-11T12:36:03.722 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-11T12:36:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.828+0000 7fc5b8699640 1 -- 172.21.15.47:0/2668539826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 msgr2=0x7fc5b010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.828+0000 7fc5b8699640 1 --2- 172.21.15.47:0/2668539826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b010abf0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009ca0 tx=0x7fc5a002a8c0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- 172.21.15.47:0/2668539826 shutdown_connections 2024-05-11T12:36:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 --2- 172.21.15.47:0/2668539826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b010abf0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 --2- 172.21.15.47:0/2668539826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b0105ed0 0x7fc5b01082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 --2- 172.21.15.47:0/2668539826 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b01035a0 0x7fc5b0105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- 172.21.15.47:0/2668539826 >> 172.21.15.47:0/2668539826 conn(0x7fc5b00fcf30 msgr2=0x7fc5b00ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- 172.21.15.47:0/2668539826 shutdown_connections 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- 172.21.15.47:0/2668539826 wait complete. 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 Processor -- start 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- start start 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b01035a0 0x7fc5b0198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b0105ed0 0x7fc5b0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b01a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b010e5f0 con 0x7fc5b0108800 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b010e470 con 0x7fc5b01035a0 2024-05-11T12:36:03.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b8699640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b010e770 con 0x7fc5b0105ed0 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b6c36640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b01a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b6c36640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b01a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46636/0 (socket says 172.21.15.47:46636) 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.829+0000 7fc5b6c36640 1 -- 172.21.15.47:0/965832508 learned_addr learned my addr 172.21.15.47:0/965832508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6435640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b01035a0 0x7fc5b0198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b5c34640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b0105ed0 0x7fc5b0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6c36640 1 -- 172.21.15.47:0/965832508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b0105ed0 msgr2=0x7fc5b0199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6c36640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b0105ed0 0x7fc5b0199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6c36640 1 -- 172.21.15.47:0/965832508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b01035a0 msgr2=0x7fc5b0198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6c36640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b01035a0 0x7fc5b0198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6c36640 1 -- 172.21.15.47:0/965832508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b01a0920 con 0x7fc5b0108800 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6435640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b01035a0 0x7fc5b0198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b6c36640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b01a0280 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00097b0 tx=0x7fc5a0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:03.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5af7fe640 1 -- 172.21.15.47:0/965832508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0004240 con 0x7fc5b0108800 2024-05-11T12:36:03.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5af7fe640 1 -- 172.21.15.47:0/965832508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a00317c0 con 0x7fc5b0108800 2024-05-11T12:36:03.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b01a0bb0 con 0x7fc5b0108800 2024-05-11T12:36:03.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5af7fe640 1 -- 172.21.15.47:0/965832508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a002fd80 con 0x7fc5b0108800 2024-05-11T12:36:03.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.830+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a1150 con 0x7fc5b0108800 2024-05-11T12:36:03.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.831+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b00fea20 con 0x7fc5b0108800 2024-05-11T12:36:03.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.832+0000 7fc5af7fe640 1 -- 172.21.15.47:0/965832508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc5a00043e0 con 0x7fc5b0108800 2024-05-11T12:36:03.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.833+0000 7fc5af7fe640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc5980b2870 0x7fc5980b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:03.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.834+0000 7fc5b6435640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc5980b2870 0x7fc5980b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:03.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.834+0000 7fc5af7fe640 1 -- 172.21.15.47:0/965832508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fc5a0103db0 con 0x7fc5b0108800 2024-05-11T12:36:03.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.834+0000 7fc5b6435640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc5980b2870 0x7fc5980b4d30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc5b019a1e0 tx=0x7fc5a8005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:03.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:03.838+0000 7fc5af7fe640 1 -- 172.21.15.47:0/965832508 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5a00cb9a0 con 0x7fc5b0108800 2024-05-11T12:36:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:03 smithi047 ceph-mon[30375]: pgmap v743: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 265 MiB/s wr, 985 op/s 2024-05-11T12:36:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:03 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1659772342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:36:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:03 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/271426420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:36:04.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:03 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3576055248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T12:36:04.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.032+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc5b0063660 con 0x7fc5b0108800 2024-05-11T12:36:04.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.033+0000 7fc5af7fe640 1 -- 172.21.15.47:0/965832508 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fc5a00098b0 con 0x7fc5b0108800 2024-05-11T12:36:04.034 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.034+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc5980b2870 msgr2=0x7fc5980b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.034+0000 7fc5b8699640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc5980b2870 0x7fc5980b4d30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc5b019a1e0 tx=0x7fc5a8005e70 comp rx=0 tx=0).stop 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.034+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 msgr2=0x7fc5b01a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.034+0000 7fc5b8699640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b01a0280 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00097b0 tx=0x7fc5a0003c20 comp rx=0 tx=0).stop 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 shutdown_connections 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fc5980b2870 0x7fc5980b4d30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc5b0108800 0x7fc5b01a0280 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fc5b0105ed0 0x7fc5b0199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 --2- 172.21.15.47:0/965832508 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc5b01035a0 0x7fc5b0198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 >> 172.21.15.47:0/965832508 conn(0x7fc5b00fcf30 msgr2=0x7fc5b0104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 shutdown_connections 2024-05-11T12:36:04.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.035+0000 7fc5b8699640 1 -- 172.21.15.47:0/965832508 wait complete. 2024-05-11T12:36:04.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-11T12:36:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.151+0000 7f4561f9d640 1 -- 172.21.15.47:0/4233087718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c105e70 msgr2=0x7f455c108260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:04.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 --2- 172.21.15.47:0/4233087718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c105e70 0x7f455c108260 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4544009ca0 tx=0x7f454402a8c0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 -- 172.21.15.47:0/4233087718 shutdown_connections 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 --2- 172.21.15.47:0/4233087718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f455c1087a0 0x7f455c10abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 --2- 172.21.15.47:0/4233087718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c105e70 0x7f455c108260 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 --2- 172.21.15.47:0/4233087718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f455c103540 0x7f455c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 -- 172.21.15.47:0/4233087718 >> 172.21.15.47:0/4233087718 conn(0x7f455c0fcf30 msgr2=0x7f455c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 -- 172.21.15.47:0/4233087718 shutdown_connections 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 -- 172.21.15.47:0/4233087718 wait complete. 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 Processor -- start 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.152+0000 7f4561f9d640 1 -- start start 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c103540 0x7f455c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f455c105e70 0x7f455c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f455c1087a0 0x7f455c1a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:04.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10e600 con 0x7f455c103540 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10e480 con 0x7f455c1087a0 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f455c10e780 con 0x7f455c105e70 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c103540 0x7f455c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c103540 0x7f455c19aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:46654/0 (socket says 172.21.15.47:46654) 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 -- 172.21.15.47:0/3661777718 learned_addr learned my addr 172.21.15.47:0/3661777718 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455affd640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f455c105e70 0x7f455c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455bfff640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f455c1087a0 0x7f455c1a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 -- 172.21.15.47:0/3661777718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f455c105e70 msgr2=0x7f455c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f455c105e70 0x7f455c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 -- 172.21.15.47:0/3661777718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f455c1087a0 msgr2=0x7f455c1a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f455c1087a0 0x7f455c1a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 -- 172.21.15.47:0/3661777718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c1a2ad0 con 0x7f455c103540 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f455b7fe640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c103540 0x7f455c19aeb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c970 tx=0x7f454c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4558ff9640 1 -- 172.21.15.47:0/3661777718 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c007bb0 con 0x7f455c103540 2024-05-11T12:36:04.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4558ff9640 1 -- 172.21.15.47:0/3661777718 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c007d50 con 0x7f455c103540 2024-05-11T12:36:04.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1a2dc0 con 0x7f455c103540 2024-05-11T12:36:04.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4558ff9640 1 -- 172.21.15.47:0/3661777718 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c00f450 con 0x7f455c103540 2024-05-11T12:36:04.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.153+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1a31d0 con 0x7f455c103540 2024-05-11T12:36:04.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.154+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f455c10a2f0 con 0x7f455c103540 2024-05-11T12:36:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.156+0000 7f4558ff9640 1 -- 172.21.15.47:0/3661777718 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f454c004f00 con 0x7f455c103540 2024-05-11T12:36:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.157+0000 7f4558ff9640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f453c0b27f0 0x7f453c0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.157+0000 7f455affd640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f453c0b27f0 0x7f453c0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.157+0000 7f4558ff9640 1 -- 172.21.15.47:0/3661777718 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f454c0e24f0 con 0x7f455c103540 2024-05-11T12:36:04.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.157+0000 7f455affd640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f453c0b27f0 0x7f453c0b4cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4544000f80 tx=0x7f45440093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:04.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.161+0000 7f4558ff9640 1 -- 172.21.15.47:0/3661777718 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f454c014030 con 0x7f455c103540 2024-05-11T12:36:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:03 smithi050 ceph-mon[34072]: pgmap v743: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 265 MiB/s wr, 985 op/s 2024-05-11T12:36:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:03 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1659772342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:36:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:03 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/271426420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:36:04.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:03 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3576055248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T12:36:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:03 smithi064 ceph-mon[34537]: pgmap v743: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 265 MiB/s wr, 985 op/s 2024-05-11T12:36:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:03 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1659772342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:36:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:03 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/271426420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-11T12:36:04.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:03 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3576055248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-11T12:36:04.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.404+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f455c063700 con 0x7f455c103540 2024-05-11T12:36:04.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.413+0000 7f4558ff9640 1 -- 172.21.15.47:0/3661777718 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+815638 (secure 0 0 0) 0x7f454c0aa140 con 0x7f455c103540 2024-05-11T12:36:04.419 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "epoch": 18, 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "active_gid": 14152, 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "active_name": "x", 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addrs": { 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6800", 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 811043456 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.420 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:6801", 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 811043456 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "active_addr": "172.21.15.47:6801/811043456", 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "active_change": "2024-05-11T12:13:41.429919+0000", 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "active_mgr_features": 4540701547738038271, 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "available": true, 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "standbys": [ 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24112, 2024-05-11T12:36:04.421 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "z", 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.422 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.423 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.424 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.425 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.426 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.428 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.429 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-05-11T12:36:04.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.431 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.432 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-05-11T12:36:04.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-05-11T12:36:04.435 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.437 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.438 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.439 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-05-11T12:36:04.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-05-11T12:36:04.441 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-05-11T12:36:04.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-05-11T12:36:04.443 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-05-11T12:36:04.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-05-11T12:36:04.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.446 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.447 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.448 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.449 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.450 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.451 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.452 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.453 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-05-11T12:36:04.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-05-11T12:36:04.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-05-11T12:36:04.456 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.457 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.458 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.459 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.461 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.462 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.463 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.465 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-11T12:36:04.466 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.468 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-05-11T12:36:04.469 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.470 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.472 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-11T12:36:04.473 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.474 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.475 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.476 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-05-11T12:36:04.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.478 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.479 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.482 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-05-11T12:36:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-05-11T12:36:04.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.486 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.487 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.488 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.490 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.491 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.492 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.493 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.494 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-05-11T12:36:04.495 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-05-11T12:36:04.496 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-05-11T12:36:04.497 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.498 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:04.499 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.500 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.501 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-05-11T12:36:04.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-05-11T12:36:04.503 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.505 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-05-11T12:36:04.506 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.509 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.510 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.511 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.513 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-05-11T12:36:04.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-05-11T12:36:04.517 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.518 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.519 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.521 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.522 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-05-11T12:36:04.523 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.524 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.526 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.527 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.529 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.530 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.531 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.533 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.535 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.536 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.537 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.538 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.540 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.541 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-05-11T12:36:04.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-11T12:36:04.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.545 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-11T12:36:04.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.550 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-11T12:36:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-11T12:36:04.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.556 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.557 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.558 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.559 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.562 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.563 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-11T12:36:04.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.566 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-11T12:36:04.567 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-11T12:36:04.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-11T12:36:04.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.570 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.572 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-11T12:36:04.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-11T12:36:04.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-11T12:36:04.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-11T12:36:04.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-11T12:36:04.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-11T12:36:04.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-05-11T12:36:04.579 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.581 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.582 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.583 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.586 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.587 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.588 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.589 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.590 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-05-11T12:36:04.595 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.596 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.598 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.601 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.602 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-05-11T12:36:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-05-11T12:36:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-05-11T12:36:04.605 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.606 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.608 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.610 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.612 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-05-11T12:36:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-05-11T12:36:04.615 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.616 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.617 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.618 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-05-11T12:36:04.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.621 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.624 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.628 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-05-11T12:36:04.632 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-11T12:36:04.634 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.635 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.637 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.638 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.640 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-05-11T12:36:04.641 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.642 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.643 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.645 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.646 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-05-11T12:36:04.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.649 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.650 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.653 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.655 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-05-11T12:36:04.659 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.660 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.661 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.662 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.663 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.664 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.666 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.667 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.669 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.670 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.671 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.672 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.673 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.674 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.675 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.677 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.678 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.679 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.680 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.683 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.684 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.686 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.687 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.688 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.689 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.690 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.691 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.692 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.693 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-05-11T12:36:04.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-05-11T12:36:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-05-11T12:36:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.697 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-05-11T12:36:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.698 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.699 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.700 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.701 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-05-11T12:36:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-11T12:36:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-11T12:36:04.702 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.703 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.704 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.705 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.706 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.707 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.708 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.710 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.711 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.713 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.714 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.716 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.717 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-05-11T12:36:04.718 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.719 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.721 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.722 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.723 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.726 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.727 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.728 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.729 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.730 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.731 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.732 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.733 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.735 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.736 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.737 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.738 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.739 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.741 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.742 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.744 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.745 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.746 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.747 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.749 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.750 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.751 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.752 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.753 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.754 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.755 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.756 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-05-11T12:36:04.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.759 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.760 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.761 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.762 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.763 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.765 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.766 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-05-11T12:36:04.767 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.768 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.769 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.770 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.771 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.772 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.773 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.774 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-05-11T12:36:04.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-11T12:36:04.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.778 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.779 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.780 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.781 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.782 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-05-11T12:36:04.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.784 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.785 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.786 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.787 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.788 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.790 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.791 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.792 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.793 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.794 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-05-11T12:36:04.795 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.797 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.798 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.799 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.800 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.801 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.802 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.803 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.804 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.805 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:04.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.807 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.808 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.810 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.811 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.813 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.814 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.815 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.817 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.818 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.820 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.823 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.824 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.825 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.826 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.827 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.828 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.829 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.830 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.831 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.832 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.833 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-05-11T12:36:04.834 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.835 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.837 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.838 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.839 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.841 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.842 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.843 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.844 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.845 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smb", 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.846 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.847 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.848 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.849 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.850 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.851 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.852 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "update_orchestration": { 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "update_orchestration", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.853 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-05-11T12:36:04.854 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.855 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.856 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.857 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.858 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.860 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.861 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.862 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.863 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.865 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.866 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.867 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.868 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-05-11T12:36:04.869 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.870 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.871 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.872 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.873 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.874 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.875 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.876 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.877 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.878 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.879 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.880 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.881 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.882 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.883 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.884 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.885 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.886 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-05-11T12:36:04.887 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-05-11T12:36:04.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.889 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.890 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.891 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.893 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.896 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.901 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-05-11T12:36:04.902 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.904 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.905 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.906 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.907 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.908 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.909 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.910 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.911 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.912 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.913 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.914 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.915 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.916 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-05-11T12:36:04.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.918 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.919 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.920 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.922 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.923 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.925 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.926 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.927 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.928 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.929 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.931 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.932 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.933 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.934 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.935 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.936 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.937 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.938 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.939 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-05-11T12:36:04.940 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.941 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.942 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.943 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.944 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.945 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.946 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.947 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.948 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.950 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.951 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.952 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.953 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "gid": 24122, 2024-05-11T12:36:04.954 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "y", 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr_features": 4540701547738038271, 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:04.955 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.957 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.958 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.961 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.962 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-05-11T12:36:04.963 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.964 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.965 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.966 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.968 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.969 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-05-11T12:36:04.970 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.971 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-05-11T12:36:04.972 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T12:36:04.974 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.975 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:04.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.977 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-05-11T12:36:04.978 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.979 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.980 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-05-11T12:36:04.981 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.982 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.983 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.985 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.986 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:04.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.990 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.991 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.992 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.993 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.994 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.995 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-05-11T12:36:04.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.997 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:04.999 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-11T12:36:05.000 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-11T12:36:05.001 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-11T12:36:05.002 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-05-11T12:36:05.003 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-05-11T12:36:05.004 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.006 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-05-11T12:36:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-05-11T12:36:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-11T12:36:05.008 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.009 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.010 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.011 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.012 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.013 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.014 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.015 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-05-11T12:36:05.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-05-11T12:36:05.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-05-11T12:36:05.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.020 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.021 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.022 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.023 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-05-11T12:36:05.024 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-05-11T12:36:05.025 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.026 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.027 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-05-11T12:36:05.028 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.030 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.031 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.032 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.033 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.034 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.035 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.036 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-11T12:36:05.037 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.038 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.039 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.040 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.041 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.042 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.043 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.044 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.045 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.046 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.047 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.048 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.049 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-05-11T12:36:05.050 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.051 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.052 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-11T12:36:05.053 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.054 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.055 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.056 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.057 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.058 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.059 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.060 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.061 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-05-11T12:36:05.062 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-05-11T12:36:05.063 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-05-11T12:36:05.064 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-05-11T12:36:05.065 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.066 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-11T12:36:05.067 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.068 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.069 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.070 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.071 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.072 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.073 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.074 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.075 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.076 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.077 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.078 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.079 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.080 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.083 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.084 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.085 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.086 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.087 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.088 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.089 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.090 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.091 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.092 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.093 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.094 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.095 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.096 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.097 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.098 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.099 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.100 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.101 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.102 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.103 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.104 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.105 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-11T12:36:05.106 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-11T12:36:05.107 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.108 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-11T12:36:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-11T12:36:05.109 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.110 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.111 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-11T12:36:05.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.114 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.115 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.116 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.117 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.118 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.119 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.120 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.121 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.122 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.123 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.124 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-11T12:36:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-11T12:36:05.125 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.126 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.127 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.128 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-05-11T12:36:05.129 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.130 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.131 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.132 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.133 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.134 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.135 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.136 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.137 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.138 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.139 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.140 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.141 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.142 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.143 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.144 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.145 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-05-11T12:36:05.146 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-05-11T12:36:05.147 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.148 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.149 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.150 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-05-11T12:36:05.151 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.152 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.153 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.154 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.155 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.156 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.157 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.158 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.159 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.160 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.161 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.162 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-05-11T12:36:05.163 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-11T12:36:05.164 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.165 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-05-11T12:36:05.166 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.167 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.168 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.169 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.170 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.171 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.172 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.173 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-05-11T12:36:05.174 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-05-11T12:36:05.175 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.176 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.177 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.178 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.179 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.180 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.181 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.182 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.183 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.184 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.185 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.186 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.187 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.188 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.189 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.190 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.191 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-11T12:36:05.192 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.193 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T12:36:05.194 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.195 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.196 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.197 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.198 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.199 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.200 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.201 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.202 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.203 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.204 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.205 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.206 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.207 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.208 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.209 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.210 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.211 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.212 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.213 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.214 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.215 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.216 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.217 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.218 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.219 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.220 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.221 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.222 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.223 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.224 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.225 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-05-11T12:36:05.226 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.227 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.228 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.229 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.230 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.231 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.232 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.233 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.235 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.236 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.237 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.238 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.239 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.240 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.241 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.242 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.243 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.244 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.245 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.246 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.247 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.248 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.249 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.250 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.251 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.252 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.253 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.254 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-05-11T12:36:05.255 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.256 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.257 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.258 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.259 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.260 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.261 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.262 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.263 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.264 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.265 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.266 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.267 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.268 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.269 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.270 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-05-11T12:36:05.271 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.272 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.273 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.274 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.275 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.276 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.277 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.278 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.279 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.280 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-05-11T12:36:05.281 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.282 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.283 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.284 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.285 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.286 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.287 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.288 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.289 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.291 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.292 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.293 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.294 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.295 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.297 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.298 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.299 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.301 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.302 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-05-11T12:36:05.303 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.304 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:05.305 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.306 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-11T12:36:05.307 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.308 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-05-11T12:36:05.309 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-05-11T12:36:05.310 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.311 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.312 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.313 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.314 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.315 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.316 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.317 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.318 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.319 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.320 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.321 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.322 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.323 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.324 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-05-11T12:36:05.325 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.326 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.327 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.328 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.329 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.330 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.331 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.332 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.333 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.334 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.335 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.336 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.337 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.338 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.339 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.340 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.341 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.342 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.343 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.344 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.345 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.346 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.347 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.348 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.349 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.350 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.351 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.352 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.353 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.354 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.355 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.356 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.357 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.358 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.359 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.360 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.361 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.362 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-05-11T12:36:05.363 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-05-11T12:36:05.364 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-05-11T12:36:05.365 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.366 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.367 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.368 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.369 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.370 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.371 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smb", 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.372 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.373 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.374 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.375 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.376 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.377 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.378 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.379 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "update_orchestration": { 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "update_orchestration", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.380 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.381 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.382 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-05-11T12:36:05.383 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.384 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.385 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.386 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.387 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.388 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.389 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.390 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.391 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.392 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.393 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.394 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.395 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.396 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.397 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.398 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.399 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.400 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.401 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.402 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.403 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.404 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.405 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.406 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.407 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.408 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.409 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.410 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.411 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.412 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.413 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.414 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.415 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.416 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.417 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.418 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-05-11T12:36:05.419 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-05-11T12:36:05.420 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.421 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.422 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.423 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.424 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.425 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.426 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-05-11T12:36:05.427 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-05-11T12:36:05.428 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.429 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.430 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.431 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.432 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.433 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.434 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.435 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.436 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.437 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.438 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.439 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-11T12:36:05.440 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-05-11T12:36:05.441 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.442 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.443 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.444 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.445 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.446 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.447 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.448 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.449 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.450 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.451 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.452 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.453 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.454 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.455 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.456 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.457 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-05-11T12:36:05.458 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.459 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.460 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:05.461 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-05-11T12:36:05.462 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.463 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.464 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.465 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.466 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.467 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.468 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.469 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.470 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.471 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.472 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.473 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.474 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.475 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: "modules": [ 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard", 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: "iostat", 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: "nfs", 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: "restful" 2024-05-11T12:36:05.476 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "available_modules": [ 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "alerts", 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.477 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How frequently to reexamine health status", 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.478 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.479 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.480 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.482 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.483 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_destination": { 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_destination", 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.484 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email address to send alerts to", 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_from_name": { 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_from_name", 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.485 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "Ceph", 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Email From: name", 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_host": { 2024-05-11T12:36:05.486 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_host", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP server", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.487 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_password": { 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_password", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Password to authenticate with", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.488 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_port": { 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_port", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "465", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP port", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.489 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_sender": { 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_sender", 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.490 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SMTP envelope sender", 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_ssl": { 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_ssl", 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.491 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.492 INFO:teuthology.orchestra.run.smithi047.stdout: "smtp_user": { 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smtp_user", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "User to authenticate as", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.493 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.494 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "balancer", 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.495 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "active": { 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "active", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically balance PGs across cluster", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.496 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_time": { 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_time", 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0000", 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "beginning time of day to automatically balance", 2024-05-11T12:36:05.497 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "begin_weekday": { 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "begin_weekday", 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-05-11T12:36:05.498 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_max_iterations": { 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_max_iterations", 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "25", 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:05.499 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "250", 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_metrics": { 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_metrics", 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.500 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "pgs,objects,bytes", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "crush_compat_step": { 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crush_compat_step", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.501 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.5", 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0.001", 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "0.999", 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "aggressiveness of optimization", 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "end_time": { 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_time", 2024-05-11T12:36:05.502 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2359", 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ending time of day to automatically balance", 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.503 INFO:teuthology.orchestra.run.smithi047.stdout: "end_weekday": { 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "end_weekday", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "0", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "6", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-11T12:36:05.504 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.505 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.506 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.507 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.508 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.509 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.510 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min_score": { 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_score", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.511 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "upmap", 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.512 INFO:teuthology.orchestra.run.smithi047.stdout: "crush-compat", 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "none", 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "read", 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap", 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap-read" 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Balancer mode", 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_ids": { 2024-05-11T12:36:05.513 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_ids", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.514 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-11T12:36:05.515 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.516 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_deviation": { 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_deviation", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.517 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "upmap_max_optimizations": { 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "upmap_max_optimizations", 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.518 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm", 2024-05-11T12:36:05.519 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_down_multiplier": { 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_down_multiplier", 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.520 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_refresh_rate": { 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_refresh_rate", 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.521 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "20", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "agent_starting_port": { 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "agent_starting_port", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.522 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4721", 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_ptrace": { 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_ptrace", 2024-05-11T12:36:05.523 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_interval": { 2024-05-11T12:36:05.524 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_interval", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.525 INFO:teuthology.orchestra.run.smithi047.stdout: "autotune_memory_target_ratio": { 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "autotune_memory_target_ratio", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0.7", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.526 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm_log_destination": { 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cephadm_log_destination", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "file", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "file,syslog", 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: "syslog" 2024-05-11T12:36:05.527 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "cgroups_split": { 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cgroups_split", 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.528 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "config_checks_enabled": { 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_checks_enabled", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.529 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "config_dashboard": { 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_dashboard", 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.530 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_alertmanager": { 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_alertmanager", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.531 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_base": { 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_base", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.532 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Container image name, without the tag", 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_elasticsearch": { 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_elasticsearch", 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.533 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "elasticsearch container image", 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_grafana": { 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_grafana", 2024-05-11T12:36:05.534 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_haproxy": { 2024-05-11T12:36:05.535 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_haproxy", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "HAproxy container image", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_agent": { 2024-05-11T12:36:05.536 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_agent", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger agent container image", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.537 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_collector": { 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_collector", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger collector container image", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.538 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_jaeger_query": { 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_jaeger_query", 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.539 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Jaeger query container image", 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_keepalived": { 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_keepalived", 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.540 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Keepalived container image", 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.541 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_loki": { 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_loki", 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Loki container image", 2024-05-11T12:36:05.542 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_node_exporter": { 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_node_exporter", 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-11T12:36:05.543 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_nvmeof": { 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_nvmeof", 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.544 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Nvme-of container image", 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.545 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_prometheus": { 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_prometheus", 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Prometheus container image", 2024-05-11T12:36:05.546 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_promtail": { 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_promtail", 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.547 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Promtail container image", 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_samba": { 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_samba", 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-11T12:36:05.548 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Samba/SMB container image", 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "container_image_snmp_gateway": { 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_image_snmp_gateway", 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.549 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "SNMP Gateway container image", 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "container_init": { 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "container_init", 2024-05-11T12:36:05.550 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Run podman/docker with `--init`", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "daemon_cache_timeout": { 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "daemon_cache_timeout", 2024-05-11T12:36:05.551 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.552 INFO:teuthology.orchestra.run.smithi047.stdout: "default_cephadm_command_timeout": { 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_cephadm_command_timeout", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "900", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.553 INFO:teuthology.orchestra.run.smithi047.stdout: "default_registry": { 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "default_registry", 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "docker.io", 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-11T12:36:05.554 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "device_cache_timeout": { 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_cache_timeout", 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1800", 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.555 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache device inventory", 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "device_enhanced_scan": { 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_enhanced_scan", 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.556 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "facts_cache_timeout": { 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "facts_cache_timeout", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.557 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "seconds to cache host facts data", 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "grafana_dashboards_path": { 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "grafana_dashboards_path", 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.558 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "host_check_interval": { 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "host_check_interval", 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.559 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to perform a host check", 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "hw_monitoring": { 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hw_monitoring", 2024-05-11T12:36:05.560 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "inventory_list_all": { 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "inventory_list_all", 2024-05-11T12:36:05.561 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.562 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "log_refresh_metadata": { 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_refresh_metadata", 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.564 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.565 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.566 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.567 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.568 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-11T12:36:05.569 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.570 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "*", 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.571 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max_count_per_host": { 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_count_per_host", 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.572 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of daemons per service per host", 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "max_osd_draining_count": { 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_osd_draining_count", 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.573 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "migration_current": { 2024-05-11T12:36:05.574 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "migration_current", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "internal - do not modify", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.575 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "mode": { 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mode", 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "root", 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm-package", 2024-05-11T12:36:05.576 INFO:teuthology.orchestra.run.smithi047.stdout: "root" 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "mode for remote execution of cephadm", 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "oob_default_addr": { 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "oob_default_addr", 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.577 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "169.254.1.1", 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "prometheus_alerts_path": { 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus_alerts_path", 2024-05-11T12:36:05.578 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.579 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_insecure": { 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_insecure", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.580 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_password": { 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_password", 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.581 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_url": { 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_url", 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.582 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "registry_username": { 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "registry_username", 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.583 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "secure_monitoring_stack": { 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "secure_monitoring_stack", 2024-05-11T12:36:05.584 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.585 INFO:teuthology.orchestra.run.smithi047.stdout: "service_discovery_port": { 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "service_discovery_port", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8765", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "cephadm service discovery port", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.586 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.587 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "ssh_config_file": { 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssh_config_file", 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.588 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "use_agent": { 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_agent", 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.589 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "use_repo_digest": { 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "use_repo_digest", 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.590 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_failed_host_check": { 2024-05-11T12:36:05.591 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_failed_host_check", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if the host check fails", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.592 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_daemons": { 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_daemons", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.593 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_on_stray_hosts": { 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_on_stray_hosts", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.594 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crash", 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.595 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.596 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.597 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.598 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.599 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.600 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "retain_interval": { 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retain_interval", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.601 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "31536000", 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.602 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_recent_interval": { 2024-05-11T12:36:05.603 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_recent_interval", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1209600", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.604 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dashboard", 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.605 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_HOST": { 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.606 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.607 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_ENABLED": { 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_ENABLED", 2024-05-11T12:36:05.608 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.609 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.610 INFO:teuthology.orchestra.run.smithi047.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-11T12:36:05.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f453c0b27f0 msgr2=0x7f453c0b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:05.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f453c0b27f0 0x7f453c0b4cb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4544000f80 tx=0x7f45440093f0 comp rx=0 tx=0).stop 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.611 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "ENABLE_BROWSABLE_API": { 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.612 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.613 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.614 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.615 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.616 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.617 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.618 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.619 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.620 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.621 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_PASSWORD": { 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-11T12:36:05.622 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-11T12:36:05.623 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_URL": { 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_URL", 2024-05-11T12:36:05.624 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_API_USERNAME": { 2024-05-11T12:36:05.625 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.626 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.627 INFO:teuthology.orchestra.run.smithi047.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.628 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.629 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.630 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "[]", 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.631 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "MULTICLUSTER_CONFIG": { 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "{}", 2024-05-11T12:36:05.632 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_HOST": { 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.633 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.634 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-11T12:36:05.635 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-11T12:36:05.636 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.637 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.638 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.639 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.640 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-11T12:36:05.641 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.642 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.643 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_ENABLED": { 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.644 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.645 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.646 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.647 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.648 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "45", 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ACCESS_KEY": { 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-11T12:36:05.649 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.650 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "admin", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.651 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SECRET_KEY": { 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.652 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "RGW_API_SSL_VERIFY": { 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.653 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "UNSAFE_TLS_v1_2": { 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.654 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.655 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.656 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.657 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.658 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: "cross_origin_url": { 2024-05-11T12:36:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cross_origin_url", 2024-05-11T12:36:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.659 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.660 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.660 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.660 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.660 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.660 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.660 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.661 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.661 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.661 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.661 INFO:teuthology.orchestra.run.smithi047.stdout: "crt_file": { 2024-05-11T12:36:05.661 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "crt_file", 2024-05-11T12:36:05.661 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.662 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.662 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.662 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.662 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.662 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.663 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.663 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.663 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.663 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.663 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.663 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.664 INFO:teuthology.orchestra.run.smithi047.stdout: "debug": { 2024-05-11T12:36:05.664 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "debug", 2024-05-11T12:36:05.664 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.664 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.664 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.665 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.665 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.665 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.665 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.665 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Enable/disable debug options", 2024-05-11T12:36:05.665 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.666 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.666 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.666 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.666 INFO:teuthology.orchestra.run.smithi047.stdout: "jwt_token_ttl": { 2024-05-11T12:36:05.666 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "jwt_token_ttl", 2024-05-11T12:36:05.666 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.667 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.667 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.667 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "28800", 2024-05-11T12:36:05.667 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.667 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.667 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.667 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.668 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.669 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.670 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.671 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.672 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.673 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.674 INFO:teuthology.orchestra.run.smithi047.stdout: "motd": { 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "motd", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "The message of the day", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.675 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect_resolve_ip_addr": { 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "redirect_resolve_ip_addr", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.676 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.677 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8080", 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.678 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.679 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.680 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl_server_port": { 2024-05-11T12:36:05.681 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl_server_port", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8443", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.682 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "redirect", 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: "redirect" 2024-05-11T12:36:05.683 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-05-11T12:36:05.684 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "url_prefix": { 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url_prefix", 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.685 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-05-11T12:36:05.686 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_monitoring": { 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_monitoring", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "monitor device health metrics", 2024-05-11T12:36:05.687 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.688 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.689 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c103540 msgr2=0x7f455c19aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c103540 0x7f455c19aeb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f454c00c970 tx=0x7f454c00ce30 comp rx=0 tx=0).stop 2024-05-11T12:36:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 shutdown_connections 2024-05-11T12:36:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f453c0b27f0 0x7f453c0b4cb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f455c1087a0 0x7f455c1a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:05.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f455c105e70 0x7f455c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 --2- 172.21.15.47:0/3661777718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f455c103540 0x7f455c19aeb0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 >> 172.21.15.47:0/3661777718 conn(0x7f455c0fcf30 msgr2=0x7f455c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.691 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.692 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.693 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.694 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "mark_out_threshold": { 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mark_out_threshold", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "2419200", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "pool_name": { 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pool_name", 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "device_health_metrics", 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.696 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "retention_period": { 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "retention_period", 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15552000", 2024-05-11T12:36:05.697 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long to retain device health metrics", 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_frequency": { 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_frequency", 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.698 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "self_heal": { 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "self_heal", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.699 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.700 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how frequently to wake up and check device health", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.701 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "warn_threshold": { 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "warn_threshold", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.702 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7257600", 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "diskprediction_local", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.704 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.705 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.706 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.707 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.708 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.709 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "predict_interval": { 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predict_interval", 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "86400", 2024-05-11T12:36:05.710 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "predictor_model": { 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "predictor_model", 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.711 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "prophetstor", 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.712 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "600", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.713 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.714 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "influx", 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": false, 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "influxdb python module not found", 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "batch_size": { 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "batch_size", 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.715 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5000", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "database": { 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "database", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.716 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "ceph", 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.717 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": { 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "hostname", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server hostname", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.718 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "30", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "5", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.719 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.720 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.721 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.722 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.723 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.724 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.725 INFO:teuthology.orchestra.run.smithi047.stdout: "password": { 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "password", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "password of InfluxDB server user", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.726 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "port": { 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "port", 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "8086", 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "InfluxDB server port", 2024-05-11T12:36:05.727 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.728 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "ssl": { 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ssl", 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "false", 2024-05-11T12:36:05.729 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "threads": { 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threads", 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.730 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "32", 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.731 INFO:teuthology.orchestra.run.smithi047.stdout: "username": { 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "username", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "username of InfluxDB server user", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.732 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "verify_ssl": { 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "verify_ssl", 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "true", 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-11T12:36:05.733 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "insights", 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.734 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.735 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.736 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.737 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.738 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.739 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.740 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.741 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "iostat", 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.742 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.743 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.744 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.745 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.746 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.747 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.748 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "k8sevents", 2024-05-11T12:36:05.749 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_event_retention_days": { 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "ceph_event_retention_days", 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "7", 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.750 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "config_check_secs": { 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "config_check_secs", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "10", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.751 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.752 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.753 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.754 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.755 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.756 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.757 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.758 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.759 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "localpool", 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "failure_domain": { 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "failure_domain", 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "host", 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.760 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "failure domain for any created local pool", 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.761 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.762 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.763 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.764 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.765 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.766 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min_size": { 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "min_size", 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.767 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.768 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 shutdown_connections 2024-05-11T12:36:05.769 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:04 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/965832508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T12:36:05.769 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:04 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3661777718' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-11T12:36:05.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:04 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/965832508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T12:36:05.769 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:04 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3661777718' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default min_size for any created local pool", 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "num_rep": { 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "num_rep", 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.770 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3", 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default replica count for any created local pool", 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_num": { 2024-05-11T12:36:05.771 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_num", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "128", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "default pg_num for any created local pool", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "prefix": { 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prefix", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "name prefix for any created local pool", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.773 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.774 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "subtree": { 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "subtree", 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "rack", 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.775 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mds_autoscaler", 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.776 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.777 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.778 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.779 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.780 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.781 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.782 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.783 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirroring", 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.784 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.785 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.786 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.787 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.788 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.789 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.790 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.791 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "nfs", 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.792 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.793 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.794 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.795 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.796 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.797 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.798 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.799 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "fail_fs": { 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "fail_fs", 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.800 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.801 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.802 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.803 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.804 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.805 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.806 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator": { 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "orchestrator", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "cephadm", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "rook", 2024-05-11T12:36:05.807 INFO:teuthology.orchestra.run.smithi047.stdout: "test_orchestrator" 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Orchestrator backend", 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.808 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.809 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_perf_query", 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.810 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.811 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.812 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.813 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.814 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.815 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.816 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "osd_support", 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.817 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.818 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.819 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.820 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.821 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.822 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.823 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.824 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "pg_autoscaler", 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.825 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.826 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.827 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.828 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.829 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.830 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.831 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.832 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "threshold": { 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "threshold", 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.833 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "3.0", 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1.0", 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "scaling threshold", 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.834 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "progress", 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_pg_recovery_event": { 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_pg_recovery_event", 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.835 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.836 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.837 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.838 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.839 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.840 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.841 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.842 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "max_completed_events": { 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_completed_events", 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "50", 2024-05-11T12:36:05.843 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "number of past completed events to remember", 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "sleep_interval": { 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sleep_interval", 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.844 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "5", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "how long the module is going to sleep", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.845 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.846 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "prometheus", 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "cache": { 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "cache", 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.847 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "exclude_perf_counters": { 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "exclude_perf_counters", 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.848 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.849 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.849 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.849 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.849 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-11T12:36:05.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:04.420+0000 7f4561f9d640 1 -- 172.21.15.47:0/3661777718 wait complete. 2024-05-11T12:36:05.851 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-11T12:36:05.851 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.851 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.851 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.851 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.851 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.851 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.852 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.853 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.854 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.855 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.856 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools": { 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools", 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.857 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.858 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "300", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.859 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "scrape_interval": { 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "scrape_interval", 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15.0", 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.860 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "::", 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.861 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "9283", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.862 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.863 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "stale_cache_strategy": { 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stale_cache_strategy", 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.864 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "log", 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_behaviour": { 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_behaviour", 2024-05-11T12:36:05.865 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "default", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "default", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "error" 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.866 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "standby_error_status_code": { 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "standby_error_status_code", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "500", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "400", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "599", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.867 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.868 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.869 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.870 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.871 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.872 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.873 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_snap_create": { 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_snap_create", 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10", 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.874 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "mirror_snapshot_schedule": { 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "mirror_snapshot_schedule", 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.875 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.876 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "trash_purge_schedule": { 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "trash_purge_schedule", 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.877 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.878 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "restful", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "enable_auth": { 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enable_auth", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.879 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "key_file": { 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "key_file", 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.880 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.881 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.882 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.883 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.884 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.885 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.886 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "server_addr": { 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_addr", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.887 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "server_port": { 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "server_port", 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.888 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.889 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rgw", 2024-05-11T12:36:05.890 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.891 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.892 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.893 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.894 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.895 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.896 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.897 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rook", 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.898 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.899 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.900 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.901 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.902 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.903 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.904 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "storage_class": { 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "storage_class", 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "local", 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.905 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "selftest", 2024-05-11T12:36:05.906 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.907 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.908 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.909 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.910 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.911 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.913 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.913 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.913 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.914 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.914 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.914 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.914 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.914 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.914 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "roption1": { 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption1", 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.915 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "roption2": { 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "roption2", 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.916 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "xyz", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption1": { 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption1", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.917 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption2": { 2024-05-11T12:36:05.918 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption2", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption3": { 2024-05-11T12:36:05.919 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption3", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "float", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.920 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption4": { 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption4", 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.921 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption5": { 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption5", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.922 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption6": { 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption6", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.923 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "rwoption7": { 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rwoption7", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "1", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "42", 2024-05-11T12:36:05.924 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.925 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "testkey": { 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testkey", 2024-05-11T12:36:05.926 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.927 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "testlkey": { 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testlkey", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.928 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "testnewline": { 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "testnewline", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.929 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "smb", 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.930 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.931 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.932 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.933 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.934 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.935 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.936 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "update_orchestration": { 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "update_orchestration", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.937 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.938 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snap_schedule", 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "allow_m_granularity": { 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "allow_m_granularity", 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.939 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "allow minute scheduled snapshots", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "dump_on_update": { 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "dump_on_update", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.940 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "dump database to debug log on update", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.941 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.942 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.943 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.944 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.945 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.946 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.947 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "stats", 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.948 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.949 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.950 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.951 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.952 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.953 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.954 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.955 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "status", 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.956 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.957 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.958 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.959 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.960 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.961 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.962 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.963 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.964 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.965 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.966 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.967 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telegraf", 2024-05-11T12:36:05.968 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "address": { 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "address", 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.969 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-11T12:36:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.970 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "15", 2024-05-11T12:36:05.971 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.972 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.973 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.974 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.975 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.976 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.977 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.978 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.979 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.980 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.981 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "telemetry", 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_basic": { 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_basic", 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.982 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_crash": { 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_crash", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.983 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_device": { 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_device", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.984 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_ident": { 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_ident", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.985 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "channel_perf": { 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "channel_perf", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.986 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "contact": { 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "contact", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.987 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "description": { 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "description", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.988 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "device_url": { 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "device_url", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.989 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "enabled": { 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "enabled", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.990 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "24", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "8", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.991 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "last_opt_revision": { 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "last_opt_revision", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "1", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.992 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard": { 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "leaderboard_description": { 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "leaderboard_description", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.994 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.995 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.996 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:05.997 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:05.998 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "organization": { 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "organization", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:05.999 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "proxy": { 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "proxy", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.000 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.001 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "url": { 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "url", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.002 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "test_orchestrator", 2024-05-11T12:36:06.003 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:06.004 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:06.005 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.006 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.007 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.008 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.009 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:06.010 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.011 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.012 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.013 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.014 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "max_concurrent_clones": { 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "max_concurrent_clones", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "4", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.015 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "periodic_async_work": { 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "periodic_async_work", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Periodically check for async work", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.016 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_delay": { 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_delay", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.017 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "snapshot_clone_no_wait": { 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "snapshot_clone_no_wait", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "True", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.018 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.019 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix", 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "can_run": true, 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "error_string": "", 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "module_options": { 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "discovery_interval": { 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "discovery_interval", 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "uint", 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "100", 2024-05-11T12:36:06.020 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "identifier": { 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "identifier", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:06.021 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "interval": { 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "interval", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "secs", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "60", 2024-05-11T12:36:06.022 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "log_level": { 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_level", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.023 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.024 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster": { 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_cluster_level": { 2024-05-11T12:36:06.025 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_cluster_level", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "info", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [ 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "critical", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "debug", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "error", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "info", 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "warning" 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.026 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "log_to_file": { 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "log_to_file", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "bool", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "False", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.027 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "sqlite3_killpoint": { 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "sqlite3_killpoint", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "dev", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 1, 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "0", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.028 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_host": { 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_host", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.029 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_port": { 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_port", 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "int", 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "10051", 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.030 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "zabbix_sender": { 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "zabbix_sender", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "str", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "level": "advanced", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "flags": 0, 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "min": "", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "max": "", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "enum_allowed": [], 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "desc": "", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "long_desc": "", 2024-05-11T12:36:06.031 INFO:teuthology.orchestra.run.smithi047.stdout: "tags": [], 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "see_also": [] 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "services": { 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "dashboard": "https://172.21.15.47:8443/" 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "always_on_modules": { 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "octopus": [ 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-05-11T12:36:06.032 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "pacific": [ 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-05-11T12:36:06.033 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "quincy": [ 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-05-11T12:36:06.034 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "reef": [ 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: ], 2024-05-11T12:36:06.035 INFO:teuthology.orchestra.run.smithi047.stdout: "squid": [ 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "balancer", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "crash", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "devicehealth", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "orchestrator", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "pg_autoscaler", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "progress", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "rbd_support", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "status", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "telemetry", 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "volumes" 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "last_failure_osd_epoch": 3, 2024-05-11T12:36:06.036 INFO:teuthology.orchestra.run.smithi047.stdout: "active_clients": [ 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "devicehealth", 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 91951901 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "libcephsqlite", 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.037 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4119076698 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "rbd_support", 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3680207492 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.038 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "name": "volumes", 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addrvec": [ 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v2", 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3079808889 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-05-11T12:36:06.039 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-05-11T12:36:06.046 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-11T12:36:06.056 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-11T12:36:06.056 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:36:06.056 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:36:06.056 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:04 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/965832508' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-11T12:36:06.057 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:04 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3661777718' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-11T12:36:06.088 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-05-11T12:36:06.088 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-11T12:36:06.150 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd blocklist ls 2024-05-11T12:36:06.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:05 smithi050 ceph-mon[34072]: pgmap v744: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 123 MiB/s wr, 689 op/s 2024-05-11T12:36:06.217 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:05 smithi047 ceph-mon[30375]: pgmap v744: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 123 MiB/s wr, 689 op/s 2024-05-11T12:36:06.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:05 smithi064 ceph-mon[34537]: pgmap v744: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 123 MiB/s wr, 689 op/s 2024-05-11T12:36:06.454 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:08.167 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:07 smithi064 ceph-mon[34537]: pgmap v745: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 111 MiB/s wr, 607 op/s 2024-05-11T12:36:08.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:07 smithi050 ceph-mon[34072]: pgmap v745: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 111 MiB/s wr, 607 op/s 2024-05-11T12:36:08.211 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:07 smithi047 ceph-mon[30375]: pgmap v745: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 111 MiB/s wr, 607 op/s 2024-05-11T12:36:08.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 -- 172.21.15.47:0/175340311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca28103f60 msgr2=0x7fca28110440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/175340311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca28103f60 0x7fca28110440 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fca20009990 tx=0x7fca20009ca0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 -- 172.21.15.47:0/175340311 shutdown_connections 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/175340311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca28103f60 0x7fca28110440 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/175340311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca281096b0 0x7fca28109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/175340311 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca281035a0 0x7fca28103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 -- 172.21.15.47:0/175340311 >> 172.21.15.47:0/175340311 conn(0x7fca2807a700 msgr2=0x7fca2807ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 -- 172.21.15.47:0/175340311 shutdown_connections 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.454+0000 7fca2ecc0700 1 -- 172.21.15.47:0/175340311 wait complete. 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 Processor -- start 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 -- start start 2024-05-11T12:36:08.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca281035a0 0x7fca28075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca28103f60 0x7fca28075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca281096b0 0x7fca280762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fca28113740 con 0x7fca281035a0 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fca281135c0 con 0x7fca28103f60 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ecc0700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fca281138c0 con 0x7fca281096b0 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ca5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca281035a0 0x7fca28075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ca5c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca281035a0 0x7fca28075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:45904/0 (socket says 172.21.15.47:45904) 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.455+0000 7fca2ca5c700 1 -- 172.21.15.47:0/878118348 learned_addr learned my addr 172.21.15.47:0/878118348 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2d25d700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca281096b0 0x7fca280762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca27fff700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca28103f60 0x7fca28075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2ca5c700 1 -- 172.21.15.47:0/878118348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca281096b0 msgr2=0x7fca280762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:08.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2ca5c700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca281096b0 0x7fca280762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2ca5c700 1 -- 172.21.15.47:0/878118348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca28103f60 msgr2=0x7fca28075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2ca5c700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca28103f60 0x7fca28075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2ca5c700 1 -- 172.21.15.47:0/878118348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca281a8050 con 0x7fca281035a0 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2ca5c700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca281035a0 0x7fca28075850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fca1800c9b0 tx=0x7fca1800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca25ffb700 1 -- 172.21.15.47:0/878118348 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18007b00 con 0x7fca281035a0 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca25ffb700 1 -- 172.21.15.47:0/878118348 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca18007ca0 con 0x7fca281035a0 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca281a8340 con 0x7fca281035a0 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.456+0000 7fca25ffb700 1 -- 172.21.15.47:0/878118348 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18018660 con 0x7fca281035a0 2024-05-11T12:36:08.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.457+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281a8880 con 0x7fca281035a0 2024-05-11T12:36:08.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.457+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2806a240 con 0x7fca281035a0 2024-05-11T12:36:08.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.459+0000 7fca25ffb700 1 -- 172.21.15.47:0/878118348 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fca1800f450 con 0x7fca281035a0 2024-05-11T12:36:08.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.459+0000 7fca25ffb700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fca140b2880 0x7fca140b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:08.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.459+0000 7fca27fff700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fca140b2880 0x7fca140b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:08.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.459+0000 7fca25ffb700 1 -- 172.21.15.47:0/878118348 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fca180e2250 con 0x7fca281035a0 2024-05-11T12:36:08.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.460+0000 7fca27fff700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fca140b2880 0x7fca140b4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca2807ac00 tx=0x7fca10009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:08.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.464+0000 7fca25ffb700 1 -- 172.21.15.47:0/878118348 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca18014030 con 0x7fca281035a0 2024-05-11T12:36:08.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.698+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fca2802e570 con 0x7fca281035a0 2024-05-11T12:36:08.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.698+0000 7fca25ffb700 1 -- 172.21.15.47:0/878118348 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+590 (secure 0 0 0) 0x7fca180a9ea0 con 0x7fca281035a0 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/3174346748 2024-05-12T12:13:41.429763+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/3174346748 2024-05-12T12:13:41.429763+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/892003243 2024-05-12T12:13:41.429763+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/1613929613 2024-05-12T12:13:41.429763+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/4061249581 2024-05-12T12:13:13.704186+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/4250695572 2024-05-12T12:13:13.704186+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/2978137437 2024-05-12T12:13:13.704186+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6800/2872683180 2024-05-12T12:13:13.704186+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6801/2872683180 2024-05-12T12:13:13.704186+0000 2024-05-11T12:36:08.699 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:0/684328265 2024-05-12T12:13:41.429763+0000 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fca140b2880 msgr2=0x7fca140b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fca140b2880 0x7fca140b4d40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca2807ac00 tx=0x7fca10009040 comp rx=0 tx=0).stop 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca281035a0 msgr2=0x7fca28075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca281035a0 0x7fca28075850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fca1800c9b0 tx=0x7fca1800ccc0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 shutdown_connections 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fca281096b0 0x7fca280762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca281035a0 0x7fca28075850 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fca140b2880 0x7fca140b4d40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 --2- 172.21.15.47:0/878118348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fca28103f60 0x7fca28075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:08.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 >> 172.21.15.47:0/878118348 conn(0x7fca2807a700 msgr2=0x7fca28102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:08.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 shutdown_connections 2024-05-11T12:36:08.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:08.701+0000 7fca2ecc0700 1 -- 172.21.15.47:0/878118348 wait complete. 2024-05-11T12:36:08.702 INFO:teuthology.orchestra.run.smithi047.stderr:listed 10 entries 2024-05-11T12:36:09.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:08 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/878118348' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T12:36:09.268 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:08 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/878118348' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T12:36:09.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:08 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/878118348' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-11T12:36:09.613 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-11T12:36:09.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:36:09.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-11T12:36:10.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:09 smithi050 ceph-mon[34072]: pgmap v746: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 93 MiB/s wr, 372 op/s 2024-05-11T12:36:10.277 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:09 smithi047 ceph-mon[30375]: pgmap v746: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 93 MiB/s wr, 372 op/s 2024-05-11T12:36:10.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:09 smithi064 ceph-mon[34537]: pgmap v746: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 93 MiB/s wr, 372 op/s 2024-05-11T12:36:12.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:11 smithi050 ceph-mon[34072]: pgmap v747: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 61 MiB/s wr, 107 op/s 2024-05-11T12:36:12.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:11 smithi047 ceph-mon[30375]: pgmap v747: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 61 MiB/s wr, 107 op/s 2024-05-11T12:36:12.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:11 smithi064 ceph-mon[34537]: pgmap v747: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 61 MiB/s wr, 107 op/s 2024-05-11T12:36:13.722 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-05-11T12:36:13.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:36:13.722 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-11T12:36:13.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:36:13.765 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-05-11T12:36:13.846 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-11T12:36:13.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:36:13.847 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-11T12:36:13.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-11T12:36:13.915 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-05-11T12:36:13.978 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:13 smithi047 ceph-mon[30375]: pgmap v748: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 352 op/s 2024-05-11T12:36:14.013 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-11T12:36:14.027 INFO:tasks.cephadm:Teardown begin 2024-05-11T12:36:14.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-11T12:36:14.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:36:14.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:13 smithi050 ceph-mon[34072]: pgmap v748: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 352 op/s 2024-05-11T12:36:14.309 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:14.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:13 smithi064 ceph-mon[34537]: pgmap v748: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 134 MiB/s wr, 352 op/s 2024-05-11T12:36:16.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:15 smithi050 ceph-mon[34072]: pgmap v749: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 517 op/s 2024-05-11T12:36:16.204 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:15 smithi047 ceph-mon[30375]: pgmap v749: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 517 op/s 2024-05-11T12:36:16.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:15 smithi064 ceph-mon[34537]: pgmap v749: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 517 op/s 2024-05-11T12:36:17.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 -- 172.21.15.47:0/937407034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 msgr2=0x7f1e081103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 --2- 172.21.15.47:0/937407034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e081103e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1e0000b0d0 tx=0x7f1e0000b3e0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 -- 172.21.15.47:0/937407034 shutdown_connections 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 --2- 172.21.15.47:0/937407034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e081103e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 --2- 172.21.15.47:0/937407034 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e08109530 0x7f1e08109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 --2- 172.21.15.47:0/937407034 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e08101b40 0x7f1e08101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 -- 172.21.15.47:0/937407034 >> 172.21.15.47:0/937407034 conn(0x7f1e08078ac0 msgr2=0x7f1e080ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 -- 172.21.15.47:0/937407034 shutdown_connections 2024-05-11T12:36:17.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.128+0000 7f1e10336700 1 -- 172.21.15.47:0/937407034 wait complete. 2024-05-11T12:36:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 Processor -- start 2024-05-11T12:36:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 -- start start 2024-05-11T12:36:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e08101b40 0x7f1e0819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e0819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:17.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e08109530 0x7f1e08199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08113720 con 0x7f1e08102500 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081135a0 con 0x7f1e08109530 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e10336700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f1e081138a0 con 0x7f1e08101b40 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e0d8d1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e0819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e0d8d1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e0819fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47768/0 (socket says 172.21.15.47:47768) 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.129+0000 7f1e0d8d1700 1 -- 172.21.15.47:0/805276211 learned_addr learned my addr 172.21.15.47:0/805276211 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0e0d2700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e08101b40 0x7f1e0819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0e8d3700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e08109530 0x7f1e08199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0d8d1700 1 -- 172.21.15.47:0/805276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e08101b40 msgr2=0x7f1e0819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0d8d1700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e08101b40 0x7f1e0819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0d8d1700 1 -- 172.21.15.47:0/805276211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e08109530 msgr2=0x7f1e08199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0d8d1700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e08109530 0x7f1e08199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0d8d1700 1 -- 172.21.15.47:0/805276211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0819a1c0 con 0x7f1e08102500 2024-05-11T12:36:17.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0e0d2700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e08101b40 0x7f1e0819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:36:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e0d8d1700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e0819fde0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9a0 tx=0x7f1df800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1dff7fe700 1 -- 172.21.15.47:0/805276211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800ca10 con 0x7f1e08102500 2024-05-11T12:36:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0819a4b0 con 0x7f1e08102500 2024-05-11T12:36:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1dff7fe700 1 -- 172.21.15.47:0/805276211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df800cbb0 con 0x7f1e08102500 2024-05-11T12:36:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1dff7fe700 1 -- 172.21.15.47:0/805276211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8018680 con 0x7f1e08102500 2024-05-11T12:36:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.130+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e081a6770 con 0x7f1e08102500 2024-05-11T12:36:17.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.132+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e08104e60 con 0x7f1e08102500 2024-05-11T12:36:17.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.133+0000 7f1dff7fe700 1 -- 172.21.15.47:0/805276211 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1df8010430 con 0x7f1e08102500 2024-05-11T12:36:17.134 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:16 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:36:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.134+0000 7f1dff7fe700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1df40b2880 0x7f1df40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.134+0000 7f1dff7fe700 1 -- 172.21.15.47:0/805276211 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f1df80e2430 con 0x7f1e08102500 2024-05-11T12:36:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.135+0000 7f1e0e0d2700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1df40b2880 0x7f1df40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:17.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.135+0000 7f1e0e0d2700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1df40b2880 0x7f1df40b4d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e0819aba0 tx=0x7f1e04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:17.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.143+0000 7f1dff7fe700 1 -- 172.21.15.47:0/805276211 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1df80aa080 con 0x7f1e08102500 2024-05-11T12:36:17.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:16 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:36:17.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:16 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-11T12:36:17.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.415+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e0819af90 con 0x7f1e08102500 2024-05-11T12:36:17.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.416+0000 7f1dff7fe700 1 -- 172.21.15.47:0/805276211 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f1df80b2100 con 0x7f1e08102500 2024-05-11T12:36:17.416 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:36:17.416 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1df40b2880 msgr2=0x7f1df40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1df40b2880 0x7f1df40b4d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1e0819aba0 tx=0x7f1e04009260 comp rx=0 tx=0).stop 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 msgr2=0x7f1e0819fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e0819fde0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9a0 tx=0x7f1df800ecb0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 shutdown_connections 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f1df40b2880 0x7f1df40b4d40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1e08109530 0x7f1e08199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f1e08101b40 0x7f1e0819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 --2- 172.21.15.47:0/805276211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1e08102500 0x7f1e0819fde0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.418+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 >> 172.21.15.47:0/805276211 conn(0x7f1e08078ac0 msgr2=0x7f1e081002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.419+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 shutdown_connections 2024-05-11T12:36:17.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:17.419+0000 7f1e10336700 1 -- 172.21.15.47:0/805276211 wait complete. 2024-05-11T12:36:17.420 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:36:18.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:17 smithi047 ceph-mon[30375]: pgmap v750: 97 pgs: 97 active+clean; 5.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 231 MiB/s wr, 1.03k op/s 2024-05-11T12:36:18.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:36:18.025 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:36:18.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:17 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:36:18.026 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:17 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/805276211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:36:18.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs dump --format=json 2024-05-11T12:36:18.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:17 smithi050 ceph-mon[34072]: pgmap v750: 97 pgs: 97 active+clean; 5.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 231 MiB/s wr, 1.03k op/s 2024-05-11T12:36:18.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:36:18.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:36:18.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:17 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:36:18.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:17 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/805276211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:36:18.347 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:18.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:17 smithi064 ceph-mon[34537]: pgmap v750: 97 pgs: 97 active+clean; 5.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 231 MiB/s wr, 1.03k op/s 2024-05-11T12:36:18.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-11T12:36:18.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-11T12:36:18.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:17 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' 2024-05-11T12:36:18.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:17 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/805276211' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:36:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/2308267494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441041b0 msgr2=0x7f7c441081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:20.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/2308267494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441041b0 0x7f7c441081d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7c340099c0 tx=0x7f7c34009cd0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/2308267494 shutdown_connections 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/2308267494 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c44108900 0x7f7c44110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/2308267494 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c44103800 0x7f7c44103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/2308267494 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c441041b0 0x7f7c441081d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/2308267494 >> 172.21.15.47:0/2308267494 conn(0x7f7c4407a6b0 msgr2=0x7f7c4407aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/2308267494 shutdown_connections 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.178+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/2308267494 wait complete. 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 Processor -- start 2024-05-11T12:36:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 -- start start 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c44103800 0x7f7c44075850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c441041b0 0x7f7c44075d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c44108900 0x7f7c440762d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7c441136c0 con 0x7f7c44103800 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44113540 con 0x7f7c441041b0 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4b3ee700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f7c44113840 con 0x7f7c44108900 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c44103800 0x7f7c44075850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c44103800 0x7f7c44075850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47794/0 (socket says 172.21.15.47:47794) 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 -- 172.21.15.47:0/1957266631 learned_addr learned my addr 172.21.15.47:0/1957266631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4998b700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c44108900 0x7f7c440762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c48989700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c441041b0 0x7f7c44075d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 -- 172.21.15.47:0/1957266631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c44108900 msgr2=0x7f7c440762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c44108900 0x7f7c440762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 -- 172.21.15.47:0/1957266631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c441041b0 msgr2=0x7f7c44075d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c441041b0 0x7f7c44075d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 -- 172.21.15.47:0/1957266631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c441a8140 con 0x7f7c44103800 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.179+0000 7f7c4918a700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c44103800 0x7f7c44075850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c920 tx=0x7f7c4000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.180+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/1957266631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40007b00 con 0x7f7c44103800 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.180+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/1957266631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c40004ce0 con 0x7f7c44103800 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.180+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c441a8430 con 0x7f7c44103800 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.180+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/1957266631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40005650 con 0x7f7c44103800 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.180+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c441a8940 con 0x7f7c44103800 2024-05-11T12:36:20.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.181+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4406a240 con 0x7f7c44103800 2024-05-11T12:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.182+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/1957266631 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7c40004750 con 0x7f7c44103800 2024-05-11T12:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.183+0000 7f7c3a7fc700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c300b2880 0x7f7c300b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.183+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/1957266631 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f7c400e2250 con 0x7f7c44103800 2024-05-11T12:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.183+0000 7f7c48989700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c300b2880 0x7f7c300b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:20.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.184+0000 7f7c48989700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c300b2880 0x7f7c300b4d40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7c440772e0 tx=0x7f7c3401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:20.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.187+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/1957266631 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7c40014030 con 0x7f7c44103800 2024-05-11T12:36:20.188 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:19 smithi047 ceph-mon[30375]: pgmap v751: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 288 MiB/s wr, 1.53k op/s 2024-05-11T12:36:20.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:19 smithi050 ceph-mon[34072]: pgmap v751: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 288 MiB/s wr, 1.53k op/s 2024-05-11T12:36:20.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:19 smithi064 ceph-mon[34537]: pgmap v751: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 288 MiB/s wr, 1.53k op/s 2024-05-11T12:36:20.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.447+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c44076d80 con 0x7f7c44103800 2024-05-11T12:36:20.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.448+0000 7f7c3a7fc700 1 -- 172.21.15.47:0/1957266631 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11791 (secure 0 0 0) 0x7f7c400a9ea0 con 0x7f7c44103800 2024-05-11T12:36:20.448 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:36:20.449 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3289515287","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3289515287},{"type":"v1","addr":"172.21.15.47:6835","nonce":3289515287}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14550,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6837/2341314538","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6836","nonce":2341314538},{"type":"v1","addr":"172.21.15.47:6837","nonce":2341314538}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6839/1816706003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6838","nonce":1816706003},{"type":"v1","addr":"172.21.15.47:6839","nonce":1816706003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6841/3463326526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6840","nonce":3463326526},{"type":"v1","addr":"172.21.15.47:6841","nonce":3463326526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6833/395408259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6832","nonce":395408259},{"type":"v1","addr":"172.21.15.64:6833","nonce":395408259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6835/470230978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6834","nonce":470230978},{"type":"v1","addr":"172.21.15.64:6835","nonce":470230978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6837/2460224166","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6836","nonce":2460224166},{"type":"v1","addr":"172.21.15.64:6837","nonce":2460224166}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6833/995235547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6832","nonce":995235547},{"type":"v1","addr":"172.21.15.50:6833","nonce":995235547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.64:6839/4099525243","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6838","nonce":4099525243},{"type":"v1","addr":"172.21.15.64:6839","nonce":4099525243}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24460,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.50:6839/1725807153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6838","nonce":1725807153},{"type":"v1","addr":"172.21.15.50:6839","nonce":1725807153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-11T12:21:13.603454+0000","modified":"2024-05-11T12:21:30.461773+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.50:6835/1314438872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6834","nonce":1314438872},{"type":"v1","addr":"172.21.15.50:6835","nonce":1314438872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":0,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.50:6837/3384616382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6836","nonce":3384616382},{"type":"v1","addr":"172.21.15.50:6837","nonce":3384616382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24485,"qdb_cluster":[24485]},"id":1}]} 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c300b2880 msgr2=0x7f7c300b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c300b2880 0x7f7c300b4d40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7c440772e0 tx=0x7f7c3401a040 comp rx=0 tx=0).stop 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c44103800 msgr2=0x7f7c44075850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c44103800 0x7f7c44075850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c920 tx=0x7f7c4000cce0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 shutdown_connections 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f7c44108900 0x7f7c440762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7c44103800 0x7f7c44075850 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f7c300b2880 0x7f7c300b4d40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 --2- 172.21.15.47:0/1957266631 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7c441041b0 0x7f7c44075d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:20.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 >> 172.21.15.47:0/1957266631 conn(0x7f7c4407a6b0 msgr2=0x7f7c44101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:20.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.450+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 shutdown_connections 2024-05-11T12:36:20.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:20.451+0000 7f7c4b3ee700 1 -- 172.21.15.47:0/1957266631 wait complete. 2024-05-11T12:36:20.452 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 20 2024-05-11T12:36:21.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:20 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/1957266631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:36:21.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:20 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/1957266631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:36:21.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:20 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/1957266631' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-11T12:36:22.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:21 smithi050 ceph-mon[34072]: pgmap v752: 97 pgs: 97 active+clean; 236 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 272 MiB/s wr, 1.53k op/s 2024-05-11T12:36:22.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:21 smithi047 ceph-mon[30375]: pgmap v752: 97 pgs: 97 active+clean; 236 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 272 MiB/s wr, 1.53k op/s 2024-05-11T12:36:22.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:21 smithi064 ceph-mon[34537]: pgmap v752: 97 pgs: 97 active+clean; 236 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 272 MiB/s wr, 1.53k op/s 2024-05-11T12:36:23.296 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd dump --format=json 2024-05-11T12:36:23.558 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:24.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:23 smithi047 ceph-mon[30375]: pgmap v753: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 267 MiB/s wr, 1.52k op/s 2024-05-11T12:36:24.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:23 smithi064 ceph-mon[34537]: pgmap v753: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 267 MiB/s wr, 1.52k op/s 2024-05-11T12:36:24.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:23 smithi050 ceph-mon[34072]: pgmap v753: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 267 MiB/s wr, 1.52k op/s 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 -- 172.21.15.47:0/2326644763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 msgr2=0x7f81c4110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/2326644763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 0x7f81c4110340 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00b0a0 tx=0x7f81bc00b3b0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 -- 172.21.15.47:0/2326644763 shutdown_connections 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/2326644763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 0x7f81c4110340 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/2326644763 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4108630 0x7f81c4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/2326644763 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c4101f10 0x7f81c4102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 -- 172.21.15.47:0/2326644763 >> 172.21.15.47:0/2326644763 conn(0x7f81c4077770 msgr2=0x7f81c4077b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 -- 172.21.15.47:0/2326644763 shutdown_connections 2024-05-11T12:36:25.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.272+0000 7f81cb0cb700 1 -- 172.21.15.47:0/2326644763 wait complete. 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 Processor -- start 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 -- start start 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4101f10 0x7f81c41030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 0x7f81c4103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c4108630 0x7f81c4103b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41137c0 con 0x7f81c41028d0 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4113640 con 0x7f81c4101f10 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.273+0000 7f81cb0cb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4113940 con 0x7f81c4108630 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4101f10 0x7f81c41030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4101f10 0x7f81c41030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.47:52942/0 (socket says 172.21.15.47:52942) 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 -- 172.21.15.47:0/480925746 learned_addr learned my addr 172.21.15.47:0/480925746 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:25.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c9668700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c4108630 0x7f81c4103b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81bbfff700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 0x7f81c4103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 -- 172.21.15.47:0/480925746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c4108630 msgr2=0x7f81c4103b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c4108630 0x7f81c4103b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 -- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 msgr2=0x7f81c4103630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 0x7f81c4103630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 -- 172.21.15.47:0/480925746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c41a8130 con 0x7f81c4101f10 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81bbfff700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 0x7f81c4103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.274+0000 7f81c8e67700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4101f10 0x7f81c41030f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c990 tx=0x7f81c000cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.275+0000 7f81b9ffb700 1 -- 172.21.15.47:0/480925746 <== mon.2 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0007940 con 0x7f81c4101f10 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.275+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c41a8420 con 0x7f81c4101f10 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.275+0000 7f81b9ffb700 1 -- 172.21.15.47:0/480925746 <== mon.2 v2:172.21.15.50:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c000f450 con 0x7f81c4101f10 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.275+0000 7f81b9ffb700 1 -- 172.21.15.47:0/480925746 <== mon.2 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0018660 con 0x7f81c4101f10 2024-05-11T12:36:25.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.275+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41a8960 con 0x7f81c4101f10 2024-05-11T12:36:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.276+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c406a240 con 0x7f81c4101f10 2024-05-11T12:36:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.277+0000 7f81b9ffb700 1 -- 172.21.15.47:0/480925746 <== mon.2 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f81c0007ae0 con 0x7f81c4101f10 2024-05-11T12:36:25.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.278+0000 7f81b9ffb700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f81ac0b2880 0x7f81ac0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.278+0000 7f81bbfff700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f81ac0b2880 0x7f81ac0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.278+0000 7f81b9ffb700 1 -- 172.21.15.47:0/480925746 <== mon.2 v2:172.21.15.50:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7f81c00e2210 con 0x7f81c4101f10 2024-05-11T12:36:25.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.279+0000 7f81bbfff700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f81ac0b2880 0x7f81ac0b4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81c4077c90 tx=0x7f81b4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:25.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.282+0000 7f81b9ffb700 1 -- 172.21.15.47:0/480925746 <== mon.2 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81c0014030 con 0x7f81c4101f10 2024-05-11T12:36:25.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.497+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81c41045d0 con 0x7f81c4101f10 2024-05-11T12:36:25.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.499+0000 7f81b9ffb700 1 -- 172.21.15.47:0/480925746 <== mon.2 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22917 (secure 0 0 0) 0x7f81c00a9e60 con 0x7f81c4101f10 2024-05-11T12:36:25.500 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-05-11T12:36:25.500 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":83,"fsid":"9b1a9a48-0f8f-11ef-bc98-c7b262605968","created":"2024-05-11T12:12:45.657702+0000","modified":"2024-05-11T12:22:43.640272+0000","last_up_change":"2024-05-11T12:20:06.204601+0000","last_in_change":"2024-05-11T12:19:45.624703+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-11T12:16:11.517702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-11T12:21:06.576652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-11T12:21:10.443533+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f335b8d2-f243-4dbe-862c-98d13f68228f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6803","nonce":105259815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6804","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6805","nonce":105259815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6808","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6809","nonce":105259815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":105259815},{"type":"v1","addr":"172.21.15.47:6807","nonce":105259815}]},"public_addr":"172.21.15.47:6803/105259815","cluster_addr":"172.21.15.47:6805/105259815","heartbeat_back_addr":"172.21.15.47:6809/105259815","heartbeat_front_addr":"172.21.15.47:6807/105259815","state":["exists","up"]},{"osd":1,"uuid":"1bacadec-81bc-4e48-b2f7-67645202bba0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6801","nonce":4057338653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6803","nonce":4057338653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6807","nonce":4057338653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":4057338653},{"type":"v1","addr":"172.21.15.50:6805","nonce":4057338653}]},"public_addr":"172.21.15.50:6801/4057338653","cluster_addr":"172.21.15.50:6803/4057338653","heartbeat_back_addr":"172.21.15.50:6807/4057338653","heartbeat_front_addr":"172.21.15.50:6805/4057338653","state":["exists","up"]},{"osd":2,"uuid":"7e45dbeb-0e57-4b1d-ad1d-0a0891009a7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6800","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6801","nonce":765449938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6802","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6803","nonce":765449938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6806","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6807","nonce":765449938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6804","nonce":765449938},{"type":"v1","addr":"172.21.15.64:6805","nonce":765449938}]},"public_addr":"172.21.15.64:6801/765449938","cluster_addr":"172.21.15.64:6803/765449938","heartbeat_back_addr":"172.21.15.64:6807/765449938","heartbeat_front_addr":"172.21.15.64:6805/765449938","state":["exists","up"]},{"osd":3,"uuid":"da71e86e-e235-47c9-bb5e-ebe25a5f6900","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6811","nonce":425091101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6813","nonce":425091101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6817","nonce":425091101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6814","nonce":425091101},{"type":"v1","addr":"172.21.15.47:6815","nonce":425091101}]},"public_addr":"172.21.15.47:6811/425091101","cluster_addr":"172.21.15.47:6813/425091101","heartbeat_back_addr":"172.21.15.47:6817/425091101","heartbeat_front_addr":"172.21.15.47:6815/425091101","state":["exists","up"]},{"osd":4,"uuid":"a94dfdef-2e27-4f70-b821-19770c72d177","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6809","nonce":3426956641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6811","nonce":3426956641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6815","nonce":3426956641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":3426956641},{"type":"v1","addr":"172.21.15.50:6813","nonce":3426956641}]},"public_addr":"172.21.15.50:6809/3426956641","cluster_addr":"172.21.15.50:6811/3426956641","heartbeat_back_addr":"172.21.15.50:6815/3426956641","heartbeat_front_addr":"172.21.15.50:6813/3426956641","state":["exists","up"]},{"osd":5,"uuid":"46f116be-c1a5-4e7b-b43c-3a2e79660066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6808","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6809","nonce":1906914370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6810","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6811","nonce":1906914370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6814","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6815","nonce":1906914370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6812","nonce":1906914370},{"type":"v1","addr":"172.21.15.64:6813","nonce":1906914370}]},"public_addr":"172.21.15.64:6809/1906914370","cluster_addr":"172.21.15.64:6811/1906914370","heartbeat_back_addr":"172.21.15.64:6815/1906914370","heartbeat_front_addr":"172.21.15.64:6813/1906914370","state":["exists","up"]},{"osd":6,"uuid":"0f2ec271-0e78-490a-86f4-c7058de78f6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6818","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6819","nonce":3223062314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6821","nonce":3223062314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6824","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6825","nonce":3223062314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":3223062314},{"type":"v1","addr":"172.21.15.47:6823","nonce":3223062314}]},"public_addr":"172.21.15.47:6819/3223062314","cluster_addr":"172.21.15.47:6821/3223062314","heartbeat_back_addr":"172.21.15.47:6825/3223062314","heartbeat_front_addr":"172.21.15.47:6823/3223062314","state":["exists","up"]},{"osd":7,"uuid":"b473c499-1ea3-4f0c-b534-07b725db8c14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6817","nonce":1770681841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6819","nonce":1770681841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6823","nonce":1770681841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":1770681841},{"type":"v1","addr":"172.21.15.50:6821","nonce":1770681841}]},"public_addr":"172.21.15.50:6817/1770681841","cluster_addr":"172.21.15.50:6819/1770681841","heartbeat_back_addr":"172.21.15.50:6823/1770681841","heartbeat_front_addr":"172.21.15.50:6821/1770681841","state":["exists","up"]},{"osd":8,"uuid":"e67c89b1-f9de-46be-b19d-c60f72ce48f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6816","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6817","nonce":2722012694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6818","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6819","nonce":2722012694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6822","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6823","nonce":2722012694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6820","nonce":2722012694},{"type":"v1","addr":"172.21.15.64:6821","nonce":2722012694}]},"public_addr":"172.21.15.64:6817/2722012694","cluster_addr":"172.21.15.64:6819/2722012694","heartbeat_back_addr":"172.21.15.64:6823/2722012694","heartbeat_front_addr":"172.21.15.64:6821/2722012694","state":["exists","up"]},{"osd":9,"uuid":"000a3e9c-c28a-46e4-b668-b809b80e11cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6827","nonce":2171109969}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6829","nonce":2171109969}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6833","nonce":2171109969}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2171109969},{"type":"v1","addr":"172.21.15.47:6831","nonce":2171109969}]},"public_addr":"172.21.15.47:6827/2171109969","cluster_addr":"172.21.15.47:6829/2171109969","heartbeat_back_addr":"172.21.15.47:6833/2171109969","heartbeat_front_addr":"172.21.15.47:6831/2171109969","state":["exists","up"]},{"osd":10,"uuid":"a5dc1c98-29f5-4afb-8c44-7c2085f9c414","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6825","nonce":1444033681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6827","nonce":1444033681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6831","nonce":1444033681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1444033681},{"type":"v1","addr":"172.21.15.50:6829","nonce":1444033681}]},"public_addr":"172.21.15.50:6825/1444033681","cluster_addr":"172.21.15.50:6827/1444033681","heartbeat_back_addr":"172.21.15.50:6831/1444033681","heartbeat_front_addr":"172.21.15.50:6829/1444033681","state":["exists","up"]},{"osd":11,"uuid":"8acbed28-84a5-4506-9bb4-ff7e65e8701b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6824","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6825","nonce":368350106}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6826","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6827","nonce":368350106}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6830","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6831","nonce":368350106}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.64:6828","nonce":368350106},{"type":"v1","addr":"172.21.15.64:6829","nonce":368350106}]},"public_addr":"172.21.15.64:6825/368350106","cluster_addr":"172.21.15.64:6827/368350106","heartbeat_back_addr":"172.21.15.64:6831/368350106","heartbeat_front_addr":"172.21.15.64:6829/368350106","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:19.686808+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:15:44.390685+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:08.484428+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:33.938557+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:16:59.635460+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:25.557208+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:17:52.225906+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:18.973213+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:18:44.695843+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:11.030404+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:19:37.777913+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-11T12:20:04.201306+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.b","mappings":[{"from":7,"to":4}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"2.17","mappings":[{"from":5,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":6}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.14","mappings":[{"from":1,"to":9}]},{"pgid":"3.31","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3a","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6801/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:6800/3174346748":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/892003243":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/1613929613":"2024-05-12T12:13:41.429763+0000","172.21.15.47:0/4061249581":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/4250695572":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/2978137437":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6800/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:6801/2872683180":"2024-05-12T12:13:13.704186+0000","172.21.15.47:0/684328265":"2024-05-12T12:13:41.429763+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f81ac0b2880 msgr2=0x7f81ac0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f81ac0b2880 0x7f81ac0b4d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f81c4077c90 tx=0x7f81b4009190 comp rx=0 tx=0).stop 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4101f10 msgr2=0x7f81c41030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4101f10 0x7f81c41030f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81c000c990 tx=0x7f81c000cca0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 shutdown_connections 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f81c4108630 0x7f81c4103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f81ac0b2880 0x7f81ac0b4d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f81c4101f10 0x7f81c41030f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 --2- 172.21.15.47:0/480925746 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81c41028d0 0x7f81c4103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 >> 172.21.15.47:0/480925746 conn(0x7f81c4077770 msgr2=0x7f81c41009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:25.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 shutdown_connections 2024-05-11T12:36:25.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:25.501+0000 7f81cb0cb700 1 -- 172.21.15.47:0/480925746 wait complete. 2024-05-11T12:36:26.180 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:25 smithi047 ceph-mon[30375]: pgmap v754: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 193 MiB/s wr, 1.27k op/s 2024-05-11T12:36:26.180 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:25 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/480925746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:36:26.211 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs fail cephfs 2024-05-11T12:36:26.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:25 smithi064 ceph-mon[34537]: pgmap v754: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 193 MiB/s wr, 1.27k op/s 2024-05-11T12:36:26.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:25 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/480925746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:36:26.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:25 smithi050 ceph-mon[34072]: pgmap v754: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 193 MiB/s wr, 1.27k op/s 2024-05-11T12:36:26.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:25 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/480925746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-11T12:36:26.466 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:28.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 -- 172.21.15.47:0/3645723801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581084b0 msgr2=0x7fea5806bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:28.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 --2- 172.21.15.47:0/3645723801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581084b0 0x7fea5806bd00 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fea54009990 tx=0x7fea54009ca0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 -- 172.21.15.47:0/3645723801 shutdown_connections 2024-05-11T12:36:28.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 --2- 172.21.15.47:0/3645723801 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea58105fc0 0x7fea58106420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 --2- 172.21.15.47:0/3645723801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea581084b0 0x7fea5806bd00 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 --2- 172.21.15.47:0/3645723801 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5806c240 0x7fea5806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 -- 172.21.15.47:0/3645723801 >> 172.21.15.47:0/3645723801 conn(0x7fea580794f0 msgr2=0x7fea58079900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 -- 172.21.15.47:0/3645723801 shutdown_connections 2024-05-11T12:36:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.187+0000 7fea5f356700 1 -- 172.21.15.47:0/3645723801 wait complete. 2024-05-11T12:36:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 Processor -- start 2024-05-11T12:36:28.189 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:27 smithi047 ceph-mon[30375]: pgmap v755: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 153 MiB/s wr, 1.07k op/s 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 -- start start 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5806c240 0x7fea581a02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea58105fc0 0x7fea581a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea581084b0 0x7fea581a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fea58119dc0 con 0x7fea58105fc0 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fea58119c40 con 0x7fea5806c240 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5f356700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fea58119f40 con 0x7fea581084b0 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5c8f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea58105fc0 0x7fea581a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5c8f1700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea58105fc0 0x7fea581a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51376/0 (socket says 172.21.15.47:51376) 2024-05-11T12:36:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5c8f1700 1 -- 172.21.15.47:0/2574878089 learned_addr learned my addr 172.21.15.47:0/2574878089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5d0f2700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5806c240 0x7fea581a02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.188+0000 7fea5d8f3700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea581084b0 0x7fea581a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5c8f1700 1 -- 172.21.15.47:0/2574878089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea581084b0 msgr2=0x7fea581a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5c8f1700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea581084b0 0x7fea581a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5c8f1700 1 -- 172.21.15.47:0/2574878089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5806c240 msgr2=0x7fea581a02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5c8f1700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5806c240 0x7fea581a02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5c8f1700 1 -- 172.21.15.47:0/2574878089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea581a52e0 con 0x7fea58105fc0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5d0f2700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5806c240 0x7fea581a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5c8f1700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea58105fc0 0x7fea581a0810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b530 tx=0x7fea4800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea4e7fc700 1 -- 172.21.15.47:0/2574878089 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48015020 con 0x7fea58105fc0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea581a55d0 con 0x7fea58105fc0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea4e7fc700 1 -- 172.21.15.47:0/2574878089 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea48011ce0 con 0x7fea58105fc0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.189+0000 7fea4e7fc700 1 -- 172.21.15.47:0/2574878089 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea48010d20 con 0x7fea58105fc0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.190+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581ace50 con 0x7fea58105fc0 2024-05-11T12:36:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.190+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5806a240 con 0x7fea58105fc0 2024-05-11T12:36:28.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.192+0000 7fea4e7fc700 1 -- 172.21.15.47:0/2574878089 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fea48018020 con 0x7fea58105fc0 2024-05-11T12:36:28.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.192+0000 7fea4e7fc700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fea440b2880 0x7fea440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:28.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.192+0000 7fea5d0f2700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fea440b2880 0x7fea440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:28.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.193+0000 7fea4e7fc700 1 -- 172.21.15.47:0/2574878089 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8443+0+0 (secure 0 0 0) 0x7fea480e1ad0 con 0x7fea58105fc0 2024-05-11T12:36:28.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.193+0000 7fea5d0f2700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fea440b2880 0x7fea440b4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fea54009750 tx=0x7fea5401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:28.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.197+0000 7fea4e7fc700 1 -- 172.21.15.47:0/2574878089 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea4801f020 con 0x7fea58105fc0 2024-05-11T12:36:28.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:27 smithi064 ceph-mon[34537]: pgmap v755: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 153 MiB/s wr, 1.07k op/s 2024-05-11T12:36:28.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:27 smithi050 ceph-mon[34072]: pgmap v755: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 153 MiB/s wr, 1.07k op/s 2024-05-11T12:36:28.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.458+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fea581ad020 con 0x7fea58105fc0 2024-05-11T12:36:28.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.460+0000 7fea4e7fc700 1 -- 172.21.15.47:0/2574878089 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7fea480a9720 con 0x7fea58105fc0 2024-05-11T12:36:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fea440b2880 msgr2=0x7fea440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:28.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fea440b2880 0x7fea440b4d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fea54009750 tx=0x7fea5401a040 comp rx=0 tx=0).stop 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea58105fc0 msgr2=0x7fea581a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea58105fc0 0x7fea581a0810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fea4800b530 tx=0x7fea4800b8f0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 shutdown_connections 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fea581084b0 0x7fea581a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fea440b2880 0x7fea440b4d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fea5806c240 0x7fea581a02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 --2- 172.21.15.47:0/2574878089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea58105fc0 0x7fea581a0810 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 >> 172.21.15.47:0/2574878089 conn(0x7fea580794f0 msgr2=0x7fea58104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 shutdown_connections 2024-05-11T12:36:28.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:28.464+0000 7fea5f356700 1 -- 172.21.15.47:0/2574878089 wait complete. 2024-05-11T12:36:28.466 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-11T12:36:28.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:28 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a[30371]: 2024-05-11T12:36:28.459+0000 7f773e3cc700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:36:28.949 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:36:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e[63841]: ignoring --setuser ceph since I am not root 2024-05-11T12:36:28.949 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:36:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e[63841]: ignoring --setgroup ceph since I am not root 2024-05-11T12:36:28.949 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mds[63845]: -- 172.21.15.50:0/2481248976 <== mon.2 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55664f91da40 con 0x55664f8f0c00 2024-05-11T12:36:28.949 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:36:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e[63841]: starting mds.e at 2024-05-11T12:36:28.950 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:36:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h[64152]: ignoring --setuser ceph since I am not root 2024-05-11T12:36:28.950 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:36:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h[64152]: ignoring --setgroup ceph since I am not root 2024-05-11T12:36:28.950 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mds[64156]: -- 172.21.15.50:0/2080532116 <== mon.0 v2:172.21.15.47:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560211d1da40 con 0x560211cf0800 2024-05-11T12:36:28.950 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:36:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h[64152]: starting mds.h at 2024-05-11T12:36:29.180 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:28 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2574878089' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-11T12:36:29.180 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:28 smithi047 ceph-mon[30375]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-11T12:36:29.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:28 smithi047 ceph-mon[30375]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:36:29.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:28 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2574878089' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-11T12:36:29.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:28 smithi047 ceph-mon[30375]: osdmap e84: 12 total, 12 up, 12 in 2024-05-11T12:36:29.181 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:28 smithi047 ceph-mon[30375]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-11T12:36:29.209 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-11T12:36:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:28 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2574878089' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-11T12:36:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:28 smithi064 ceph-mon[34537]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-11T12:36:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:28 smithi064 ceph-mon[34537]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:36:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:28 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2574878089' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-11T12:36:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:28 smithi064 ceph-mon[34537]: osdmap e84: 12 total, 12 up, 12 in 2024-05-11T12:36:29.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:28 smithi064 ceph-mon[34537]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-11T12:36:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2574878089' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-11T12:36:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mon[34072]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-11T12:36:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mon[34072]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-11T12:36:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2574878089' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-11T12:36:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mon[34072]: osdmap e84: 12 total, 12 up, 12 in 2024-05-11T12:36:29.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:28 smithi050 ceph-mon[34072]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-05-11T12:36:29.463 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:30.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:29 smithi047 ceph-mon[30375]: pgmap v757: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 818 B/s wr, 29 op/s 2024-05-11T12:36:30.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:29 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.50:6834/3580583270,v1:172.21.15.50:6836/3580583270] up:boot 2024-05-11T12:36:30.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:29 smithi047 ceph-mon[30375]: mds.? [v2:172.21.15.50:6835/408694230,v1:172.21.15.50:6837/408694230] up:boot 2024-05-11T12:36:30.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:29 smithi047 ceph-mon[30375]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-11T12:36:30.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T12:36:30.239 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:29 smithi047 ceph-mon[30375]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T12:36:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:29 smithi064 ceph-mon[34537]: pgmap v757: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 818 B/s wr, 29 op/s 2024-05-11T12:36:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:29 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.50:6834/3580583270,v1:172.21.15.50:6836/3580583270] up:boot 2024-05-11T12:36:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:29 smithi064 ceph-mon[34537]: mds.? [v2:172.21.15.50:6835/408694230,v1:172.21.15.50:6837/408694230] up:boot 2024-05-11T12:36:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:29 smithi064 ceph-mon[34537]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-11T12:36:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T12:36:30.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:29 smithi064 ceph-mon[34537]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T12:36:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:29 smithi050 ceph-mon[34072]: pgmap v757: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 818 B/s wr, 29 op/s 2024-05-11T12:36:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:29 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.50:6834/3580583270,v1:172.21.15.50:6836/3580583270] up:boot 2024-05-11T12:36:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:29 smithi050 ceph-mon[34072]: mds.? [v2:172.21.15.50:6835/408694230,v1:172.21.15.50:6837/408694230] up:boot 2024-05-11T12:36:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:29 smithi050 ceph-mon[34072]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-05-11T12:36:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-11T12:36:30.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:29 smithi050 ceph-mon[34072]: from='mgr.14152 172.21.15.47:0/432058683' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-11T12:36:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 -- 172.21.15.47:0/3504906699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 msgr2=0x7fef2c1081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3504906699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c1081f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fef140099c0 tx=0x7fef14009cd0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 -- 172.21.15.47:0/3504906699 shutdown_connections 2024-05-11T12:36:31.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3504906699 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef2c108920 0x7fef2c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3504906699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef2c103830 0x7fef2c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3504906699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c1081f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 -- 172.21.15.47:0/3504906699 >> 172.21.15.47:0/3504906699 conn(0x7fef2c07a6c0 msgr2=0x7fef2c07aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 -- 172.21.15.47:0/3504906699 shutdown_connections 2024-05-11T12:36:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.095+0000 7fef30ddb700 1 -- 172.21.15.47:0/3504906699 wait complete. 2024-05-11T12:36:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.096+0000 7fef30ddb700 1 Processor -- start 2024-05-11T12:36:31.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.096+0000 7fef30ddb700 1 -- start start 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef30ddb700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef2c103830 0x7fef2c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef30ddb700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef30ddb700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef2c108920 0x7fef2c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef30ddb700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c1136e0 con 0x7fef2c1041e0 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef30ddb700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113560 con 0x7fef2c103830 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef30ddb700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7fef2c113860 con 0x7fef2c108920 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51394/0 (socket says 172.21.15.47:51394) 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 -- 172.21.15.47:0/3256836694 learned_addr learned my addr 172.21.15.47:0/3256836694 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:31.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef2ad9d700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef2c108920 0x7fef2c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef2a59c700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef2c103830 0x7fef2c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 -- 172.21.15.47:0/3256836694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef2c108920 msgr2=0x7fef2c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef2c108920 0x7fef2c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 -- 172.21.15.47:0/3256836694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef2c103830 msgr2=0x7fef2c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef2c103830 0x7fef2c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 -- 172.21.15.47:0/3256836694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2c19a100 con 0x7fef2c1041e0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef2a59c700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef2c103830 0x7fef2c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.097+0000 7fef21d9b700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c19fcd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fef140098e0 tx=0x7fef140057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.098+0000 7fef23fff700 1 -- 172.21.15.47:0/3256836694 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14027070 con 0x7fef2c1041e0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.098+0000 7fef23fff700 1 -- 172.21.15.47:0/3256836694 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef14003f20 con 0x7fef2c1041e0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.098+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2c19a390 con 0x7fef2c1041e0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.098+0000 7fef23fff700 1 -- 172.21.15.47:0/3256836694 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14022da0 con 0x7fef2c1041e0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.098+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2c1a6660 con 0x7fef2c1041e0 2024-05-11T12:36:31.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.099+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef2c06a240 con 0x7fef2c1041e0 2024-05-11T12:36:31.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.100+0000 7fef23fff700 1 -- 172.21.15.47:0/3256836694 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fef1402a070 con 0x7fef2c1041e0 2024-05-11T12:36:31.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.101+0000 7fef23fff700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fef180b2900 0x7fef180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:31.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.101+0000 7fef2a59c700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fef180b2900 0x7fef180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:31.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.101+0000 7fef23fff700 1 -- 172.21.15.47:0/3256836694 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8529+0+0 (secure 0 0 0) 0x7fef140e37f0 con 0x7fef2c1041e0 2024-05-11T12:36:31.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.101+0000 7fef2a59c700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fef180b2900 0x7fef180b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fef2c108480 tx=0x7fef1c00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:31.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.104+0000 7fef23fff700 1 -- 172.21.15.47:0/3256836694 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef14031020 con 0x7fef2c1041e0 2024-05-11T12:36:31.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.372+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fef2c02e570 con 0x7fef2c1041e0 2024-05-11T12:36:31.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.967+0000 7fef23fff700 1 -- 172.21.15.47:0/3256836694 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fef14013070 con 0x7fef2c1041e0 2024-05-11T12:36:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fef180b2900 msgr2=0x7fef180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fef180b2900 0x7fef180b4dc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fef2c108480 tx=0x7fef1c00a680 comp rx=0 tx=0).stop 2024-05-11T12:36:31.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 msgr2=0x7fef2c19fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c19fcd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fef140098e0 tx=0x7fef140057c0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 shutdown_connections 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7fef2c108920 0x7fef2c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7fef180b2900 0x7fef180b4dc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fef2c103830 0x7fef2c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 --2- 172.21.15.47:0/3256836694 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef2c1041e0 0x7fef2c19fcd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 >> 172.21.15.47:0/3256836694 conn(0x7fef2c07a6c0 msgr2=0x7fef2c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 shutdown_connections 2024-05-11T12:36:31.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:31.970+0000 7fef30ddb700 1 -- 172.21.15.47:0/3256836694 wait complete. 2024-05-11T12:36:32.251 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:31 smithi050 ceph-mon[34072]: pgmap v758: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:32.251 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:31 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3256836694' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:36:32.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:31 smithi047 ceph-mon[30375]: pgmap v758: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:32.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:31 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3256836694' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:36:32.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:31 smithi064 ceph-mon[34537]: pgmap v758: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:32.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:31 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3256836694' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-11T12:36:32.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-11T12:36:33.001 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:33.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:32 smithi047 ceph-mon[30375]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-11T12:36:33.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:32 smithi047 ceph-mon[30375]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-11T12:36:33.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:32 smithi047 ceph-mon[30375]: Cluster is now healthy 2024-05-11T12:36:33.262 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:32 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3256836694' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-11T12:36:33.263 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:32 smithi047 ceph-mon[30375]: fsmap 12 up:standby 2024-05-11T12:36:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:32 smithi064 ceph-mon[34537]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-11T12:36:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:32 smithi064 ceph-mon[34537]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-11T12:36:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:32 smithi064 ceph-mon[34537]: Cluster is now healthy 2024-05-11T12:36:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:32 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3256836694' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-11T12:36:33.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:32 smithi064 ceph-mon[34537]: fsmap 12 up:standby 2024-05-11T12:36:33.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:32 smithi050 ceph-mon[34072]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-11T12:36:33.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:32 smithi050 ceph-mon[34072]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-11T12:36:33.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:32 smithi050 ceph-mon[34072]: Cluster is now healthy 2024-05-11T12:36:33.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:32 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3256836694' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-11T12:36:33.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:32 smithi050 ceph-mon[34072]: fsmap 12 up:standby 2024-05-11T12:36:34.157 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:33 smithi047 ceph-mon[30375]: pgmap v759: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:34.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:33 smithi064 ceph-mon[34537]: pgmap v759: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:34.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:33 smithi050 ceph-mon[34072]: pgmap v759: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 -- 172.21.15.47:0/2770185065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 msgr2=0x7f56001103e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 --2- 172.21.15.47:0/2770185065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f56001103e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f55f800b0a0 tx=0x7f55f800b3b0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 -- 172.21.15.47:0/2770185065 shutdown_connections 2024-05-11T12:36:34.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 --2- 172.21.15.47:0/2770185065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f56001103e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 --2- 172.21.15.47:0/2770185065 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56001095c0 0x7f56001099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 --2- 172.21.15.47:0/2770185065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560006b230 0x7f560006b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 -- 172.21.15.47:0/2770185065 >> 172.21.15.47:0/2770185065 conn(0x7f5600079400 msgr2=0x7f5600079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 -- 172.21.15.47:0/2770185065 shutdown_connections 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.714+0000 7f5606b21700 1 -- 172.21.15.47:0/2770185065 wait complete. 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 Processor -- start 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 -- start start 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560006b230 0x7f5600102520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f5600102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56001095c0 0x7f5600102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f56001136f0 con 0x7f560006bbf0 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f5600113570 con 0x7f560006b230 2024-05-11T12:36:34.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f5606b21700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f5600113870 con 0x7f56001095c0 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f55fffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f5600102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f55fffff700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f5600102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:51424/0 (socket says 172.21.15.47:51424) 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f55fffff700 1 -- 172.21.15.47:0/2245327591 learned_addr learned my addr 172.21.15.47:0/2245327591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f56048bd700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560006b230 0x7f5600102520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.715+0000 7f56050be700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56001095c0 0x7f5600102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fffff700 1 -- 172.21.15.47:0/2245327591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56001095c0 msgr2=0x7f5600102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fffff700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56001095c0 0x7f5600102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fffff700 1 -- 172.21.15.47:0/2245327591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560006b230 msgr2=0x7f5600102520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fffff700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560006b230 0x7f5600102520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fffff700 1 -- 172.21.15.47:0/2245327591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56001a80c0 con 0x7f560006bbf0 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f56050be700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56001095c0 0x7f5600102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fffff700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f5600102a60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f55f400d980 tx=0x7f55f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:34.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fdffb700 1 -- 172.21.15.47:0/2245327591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4017070 con 0x7f560006bbf0 2024-05-11T12:36:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fdffb700 1 -- 172.21.15.47:0/2245327591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55f4005020 con 0x7f560006bbf0 2024-05-11T12:36:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56001a83b0 con 0x7f560006bbf0 2024-05-11T12:36:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f55fdffb700 1 -- 172.21.15.47:0/2245327591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f4009b50 con 0x7f560006bbf0 2024-05-11T12:36:34.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.716+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56001a88a0 con 0x7f560006bbf0 2024-05-11T12:36:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.717+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560006a240 con 0x7f560006bbf0 2024-05-11T12:36:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.718+0000 7f55fdffb700 1 -- 172.21.15.47:0/2245327591 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f55f40051d0 con 0x7f560006bbf0 2024-05-11T12:36:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.719+0000 7f55fdffb700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55e80b2880 0x7f55e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.719+0000 7f56048bd700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55e80b2880 0x7f55e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.719+0000 7f55fdffb700 1 -- 172.21.15.47:0/2245327591 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8529+0+0 (secure 0 0 0) 0x7f55f40e22d0 con 0x7f560006bbf0 2024-05-11T12:36:34.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.720+0000 7f56048bd700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55e80b2880 0x7f55e80b4d40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55f0007b60 tx=0x7f55f00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:34.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.724+0000 7f55fdffb700 1 -- 172.21.15.47:0/2245327591 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f55f4020020 con 0x7f560006bbf0 2024-05-11T12:36:34.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.938+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f560002e570 con 0x7f560006bbf0 2024-05-11T12:36:34.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.975+0000 7f55fdffb700 1 -- 172.21.15.47:0/2245327591 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f55f40a9e60 con 0x7f560006bbf0 2024-05-11T12:36:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55e80b2880 msgr2=0x7f55e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55e80b2880 0x7f55e80b4d40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f55f0007b60 tx=0x7f55f00073d0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 msgr2=0x7f5600102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f5600102a60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f55f400d980 tx=0x7f55f400dc90 comp rx=0 tx=0).stop 2024-05-11T12:36:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 shutdown_connections 2024-05-11T12:36:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f56001095c0 0x7f5600102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f55e80b2880 0x7f55e80b4d40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f560006b230 0x7f5600102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 --2- 172.21.15.47:0/2245327591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f560006bbf0 0x7f5600102a60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 >> 172.21.15.47:0/2245327591 conn(0x7f5600079400 msgr2=0x7f5600101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 shutdown_connections 2024-05-11T12:36:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:34.979+0000 7f5606b21700 1 -- 172.21.15.47:0/2245327591 wait complete. 2024-05-11T12:36:34.980 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-05-11T12:36:35.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:34 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2245327591' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T12:36:35.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:34 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2245327591' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T12:36:35.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:34 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2245327591' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T12:36:35.719 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e shell --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-11T12:36:35.979 INFO:teuthology.orchestra.run.smithi047.stderr:Inferring config /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/mon.a/config 2024-05-11T12:36:36.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:35 smithi047 ceph-mon[30375]: pgmap v760: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:36.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:35 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/2245327591' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T12:36:36.238 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:35 smithi047 ceph-mon[30375]: osdmap e85: 12 total, 12 up, 12 in 2024-05-11T12:36:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:35 smithi064 ceph-mon[34537]: pgmap v760: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:35 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/2245327591' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T12:36:36.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:35 smithi064 ceph-mon[34537]: osdmap e85: 12 total, 12 up, 12 in 2024-05-11T12:36:36.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:35 smithi050 ceph-mon[34072]: pgmap v760: 97 pgs: 97 active+clean; 236 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-05-11T12:36:36.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:35 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/2245327591' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T12:36:36.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:35 smithi050 ceph-mon[34072]: osdmap e85: 12 total, 12 up, 12 in 2024-05-11T12:36:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.730+0000 7f7750e1e700 1 -- 172.21.15.47:0/1400268979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c108920 msgr2=0x7f774c1104a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.730+0000 7f7750e1e700 1 --2- 172.21.15.47:0/1400268979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c108920 0x7f774c1104a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f774400b0d0 tx=0x7f774400b3e0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 -- 172.21.15.47:0/1400268979 shutdown_connections 2024-05-11T12:36:37.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 --2- 172.21.15.47:0/1400268979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c108920 0x7f774c1104a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 --2- 172.21.15.47:0/1400268979 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c103830 0x7f774c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 --2- 172.21.15.47:0/1400268979 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c1041e0 0x7f774c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 -- 172.21.15.47:0/1400268979 >> 172.21.15.47:0/1400268979 conn(0x7f774c07a6c0 msgr2=0x7f774c07aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 -- 172.21.15.47:0/1400268979 shutdown_connections 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 -- 172.21.15.47:0/1400268979 wait complete. 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 Processor -- start 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.731+0000 7f7750e1e700 1 -- start start 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7750e1e700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c103830 0x7f774c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7750e1e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c1041e0 0x7f774c19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7750e1e700 1 --2- >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c108920 0x7f774c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7750e1e700 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f774c113770 con 0x7f774c103830 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7750e1e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1135f0 con 0x7f774c1041e0 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7750e1e700 1 -- --> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] -- mon_getmap magic: 0 -- 0x7f774c1138f0 con 0x7f774c108920 2024-05-11T12:36:37.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c103830 0x7f774c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c103830 0x7f774c19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:48960/0 (socket says 172.21.15.47:48960) 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 -- 172.21.15.47:0/3753303117 learned_addr learned my addr 172.21.15.47:0/3753303117 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7749d9b700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c1041e0 0x7f774c19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774ad9d700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c108920 0x7f774c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 -- 172.21.15.47:0/3753303117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c108920 msgr2=0x7f774c1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c108920 0x7f774c1999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 -- 172.21.15.47:0/3753303117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c1041e0 msgr2=0x7f774c19fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c1041e0 0x7f774c19fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 -- 172.21.15.47:0/3753303117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c19a110 con 0x7f774c103830 2024-05-11T12:36:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f774a59c700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c103830 0x7f774c19f7f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b530 tx=0x7f773c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f773b7fe700 1 -- 172.21.15.47:0/3753303117 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c015020 con 0x7f774c103830 2024-05-11T12:36:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f773b7fe700 1 -- 172.21.15.47:0/3753303117 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f773c011ce0 con 0x7f774c103830 2024-05-11T12:36:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f773b7fe700 1 -- 172.21.15.47:0/3753303117 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773c010dd0 con 0x7f774c103830 2024-05-11T12:36:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.732+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c19a400 con 0x7f774c103830 2024-05-11T12:36:37.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.733+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c1a66c0 con 0x7f774c103830 2024-05-11T12:36:37.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.733+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774c10bb60 con 0x7f774c103830 2024-05-11T12:36:37.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.736+0000 7f773b7fe700 1 -- 172.21.15.47:0/3753303117 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f773c018020 con 0x7f774c103830 2024-05-11T12:36:37.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.736+0000 7f773b7fe700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77340b2900 0x7f77340b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-11T12:36:37.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.737+0000 7f7749d9b700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77340b2900 0x7f77340b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-11T12:36:37.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.737+0000 7f773b7fe700 1 -- 172.21.15.47:0/3753303117 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7859+0+0 (secure 0 0 0) 0x7f773c0e1c30 con 0x7f774c103830 2024-05-11T12:36:37.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.737+0000 7f7749d9b700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77340b2900 0x7f77340b4dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f774c109c70 tx=0x7f774000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-11T12:36:37.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.740+0000 7f773b7fe700 1 -- 172.21.15.47:0/3753303117 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f773c01f020 con 0x7f774c103830 2024-05-11T12:36:37.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.957+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f774c06a240 con 0x7f774c103830 2024-05-11T12:36:37.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.978+0000 7f773b7fe700 1 -- 172.21.15.47:0/3753303117 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7f773c0a9b10 con 0x7f774c103830 2024-05-11T12:36:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.982+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77340b2900 msgr2=0x7f77340b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77340b2900 0x7f77340b4dc0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f774c109c70 tx=0x7f774000a380 comp rx=0 tx=0).stop 2024-05-11T12:36:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c103830 msgr2=0x7f774c19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-11T12:36:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c103830 0x7f774c19f7f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f773c00b530 tx=0x7f773c00b840 comp rx=0 tx=0).stop 2024-05-11T12:36:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 shutdown_connections 2024-05-11T12:36:37.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.64:3300/0,v1:172.21.15.64:6789/0] conn(0x7f774c108920 0x7f774c1999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f774c103830 0x7f774c19f7f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.47:6800/811043456,v1:172.21.15.47:6801/811043456] conn(0x7f77340b2900 0x7f77340b4dc0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 --2- 172.21.15.47:0/3753303117 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f774c1041e0 0x7f774c19fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-11T12:36:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.983+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 >> 172.21.15.47:0/3753303117 conn(0x7f774c07a6c0 msgr2=0x7f774c1015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-11T12:36:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.984+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 shutdown_connections 2024-05-11T12:36:37.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-05-11T12:36:37.984+0000 7f7750e1e700 1 -- 172.21.15.47:0/3753303117 wait complete. 2024-05-11T12:36:37.988 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-05-11T12:36:38.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:37 smithi047 ceph-mon[30375]: pgmap v762: 65 pgs: 65 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:36:38.276 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:37 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3753303117' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T12:36:38.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:37 smithi064 ceph-mon[34537]: pgmap v762: 65 pgs: 65 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:36:38.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:37 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3753303117' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T12:36:38.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:37 smithi050 ceph-mon[34072]: pgmap v762: 65 pgs: 65 active+clean; 577 KiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-11T12:36:38.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:37 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3753303117' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-11T12:36:38.821 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:36:38.852 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:36:38.936 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:36:38.996 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-11T12:36:38.996 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-11T12:36:39.015 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-11T12:36:39.046 DEBUG:teuthology.orchestra.run.smithi064:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-11T12:36:39.068 INFO:tasks.cephadm:Stopping all daemons... 2024-05-11T12:36:39.068 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-11T12:36:39.068 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a 2024-05-11T12:36:39.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:38 smithi047 ceph-mon[30375]: from='client.? 172.21.15.47:0/3753303117' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T12:36:39.094 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:38 smithi047 ceph-mon[30375]: osdmap e86: 12 total, 12 up, 12 in 2024-05-11T12:36:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:38 smithi050 ceph-mon[34072]: from='client.? 172.21.15.47:0/3753303117' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T12:36:39.199 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:38 smithi050 ceph-mon[34072]: osdmap e86: 12 total, 12 up, 12 in 2024-05-11T12:36:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:38 smithi064 ceph-mon[34537]: from='client.? 172.21.15.47:0/3753303117' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-11T12:36:39.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:38 smithi064 ceph-mon[34537]: osdmap e86: 12 total, 12 up, 12 in 2024-05-11T12:36:39.398 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:39 smithi047 systemd[1]: Stopping Ceph mon.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:39.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a[30371]: 2024-05-11T12:36:39.396+0000 7f7743bd7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:36:39.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a[30371]: 2024-05-11T12:36:39.396+0000 7f7743bd7700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-11T12:36:39.775 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:39 smithi047 podman[101534]: 2024-05-11 12:36:39.499881603 +0000 UTC m=+0.225470063 container died da3e44580bab25d4be199df2110939078700331ba83f2d22b51ea7a75c5c28c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-11T12:36:40.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:39 smithi047 podman[101534]: 2024-05-11 12:36:39.89769571 +0000 UTC m=+0.623284179 container cleanup da3e44580bab25d4be199df2110939078700331ba83f2d22b51ea7a75c5c28c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:36:40.275 INFO:journalctl@ceph.mon.a.smithi047.stdout:May 11 12:36:39 smithi047 bash[101534]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-a 2024-05-11T12:36:40.590 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.a.service' 2024-05-11T12:36:40.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:36:40.670 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-11T12:36:40.670 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-11T12:36:40.670 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.b 2024-05-11T12:36:40.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:40 smithi050 systemd[1]: Stopping Ceph mon.b for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:41.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:41 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-b[34068]: 2024-05-11T12:36:41.074+0000 7f361e7e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:36:41.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:41 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-b[34068]: 2024-05-11T12:36:41.074+0000 7f361e7e3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-11T12:36:41.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:41 smithi050 podman[68938]: 2024-05-11 12:36:41.177716736 +0000 UTC m=+0.243715747 container died cddd0d3211588557e2a54e0970637d5a7e089c5b65ca57698186cf2fe4842b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-05-11T12:36:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:41 smithi050 podman[68938]: 2024-05-11 12:36:41.68683744 +0000 UTC m=+0.752836429 container cleanup cddd0d3211588557e2a54e0970637d5a7e089c5b65ca57698186cf2fe4842b4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True) 2024-05-11T12:36:41.949 INFO:journalctl@ceph.mon.b.smithi050.stdout:May 11 12:36:41 smithi050 bash[68938]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-b 2024-05-11T12:36:42.114 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.b.service' 2024-05-11T12:36:42.159 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:36:42.159 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-11T12:36:42.159 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-11T12:36:42.159 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.c 2024-05-11T12:36:42.512 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:42 smithi064 systemd[1]: Stopping Ceph mon.c for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-c[34533]: 2024-05-11T12:36:42.510+0000 7f8b89128700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:36:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-c[34533]: 2024-05-11T12:36:42.510+0000 7f8b89128700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-11T12:36:42.864 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:42 smithi064 podman[69975]: 2024-05-11 12:36:42.705321696 +0000 UTC m=+0.340982686 container died 61ed5fe3c9ef0b11e9122e12a27ec489d40fc0b7cc58e87521eb6dcf49d99d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-c, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-11T12:36:43.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:43 smithi064 podman[69975]: 2024-05-11 12:36:43.196431722 +0000 UTC m=+0.832092761 container cleanup 61ed5fe3c9ef0b11e9122e12a27ec489d40fc0b7cc58e87521eb6dcf49d99d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-11T12:36:43.364 INFO:journalctl@ceph.mon.c.smithi064.stdout:May 11 12:36:43 smithi064 bash[69975]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mon-c 2024-05-11T12:36:43.547 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mon.c.service' 2024-05-11T12:36:43.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:36:43.600 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-11T12:36:43.600 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-11T12:36:43.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.x 2024-05-11T12:36:43.936 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:36:43 smithi047 systemd[1]: Stopping Ceph mgr.x for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:44.271 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:36:43 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x[30607]: 2024-05-11T12:36:43.935+0000 7fb7e323d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:36:44.271 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:36:43 smithi047 podman[101656]: 2024-05-11 12:36:43.967417396 +0000 UTC m=+0.164038095 container died 6c6b8a3e0be6fb68ad18ea1bb0bb67f6fe796eb231eb89add4847772b6b3ca35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-11T12:36:44.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:36:44 smithi047 podman[101656]: 2024-05-11 12:36:44.470878118 +0000 UTC m=+0.667498865 container cleanup 6c6b8a3e0be6fb68ad18ea1bb0bb67f6fe796eb231eb89add4847772b6b3ca35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:308506802b0bdea4df7050cfc1fcbdc04fea6a6e, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:36:44.525 INFO:journalctl@ceph.mgr.x.smithi047.stdout:May 11 12:36:44 smithi047 bash[101656]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-x 2024-05-11T12:36:44.898 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.x.service' 2024-05-11T12:36:44.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:36:44.936 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-11T12:36:44.936 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-11T12:36:44.936 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.y 2024-05-11T12:36:45.412 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:36:44 smithi050 systemd[1]: Stopping Ceph mgr.y for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:45.699 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:36:45 smithi050 podman[69064]: 2024-05-11 12:36:45.412740402 +0000 UTC m=+0.275245550 container died 70f85aab87bfd2d62eebac49caa8872d5a2ac79ff33d50c8acbb5da4782bfce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-05-11T12:36:46.077 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:36:45 smithi050 podman[69064]: 2024-05-11 12:36:45.817741496 +0000 UTC m=+0.680246643 container cleanup 70f85aab87bfd2d62eebac49caa8872d5a2ac79ff33d50c8acbb5da4782bfce4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:36:46.077 INFO:journalctl@ceph.mgr.y.smithi050.stdout:May 11 12:36:45 smithi050 bash[69064]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-y 2024-05-11T12:36:46.244 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.y.service' 2024-05-11T12:36:46.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:36:46.280 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-11T12:36:46.280 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-11T12:36:46.280 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.z 2024-05-11T12:36:46.614 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:36:46 smithi064 systemd[1]: Stopping Ceph mgr.z for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:46.928 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:36:46 smithi064 podman[70100]: 2024-05-11 12:36:46.622534112 +0000 UTC m=+0.140630698 container died 9ac350df9f877ae763b89070931f777914bd184b8e0bae6cca218f5ca67ff254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:36:47.364 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:36:47 smithi064 podman[70100]: 2024-05-11 12:36:47.152738277 +0000 UTC m=+0.670834862 container cleanup 9ac350df9f877ae763b89070931f777914bd184b8e0bae6cca218f5ca67ff254 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-11T12:36:47.364 INFO:journalctl@ceph.mgr.z.smithi064.stdout:May 11 12:36:47 smithi064 bash[70100]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mgr-z 2024-05-11T12:36:47.643 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mgr.z.service' 2024-05-11T12:36:47.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:36:47.678 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-11T12:36:47.678 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-11T12:36:47.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.0 2024-05-11T12:36:48.025 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:47 smithi047 systemd[1]: Stopping Ceph osd.0 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:48.525 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0[42685]: 2024-05-11T12:36:48.050+0000 7ff51078e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:36:48.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0[42685]: 2024-05-11T12:36:48.050+0000 7ff51078e700 -1 osd.0 86 *** Got signal Terminated *** 2024-05-11T12:36:48.526 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0[42685]: 2024-05-11T12:36:48.050+0000 7ff51078e700 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:36:53.903 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:53 smithi047 podman[101777]: 2024-05-11 12:36:53.574735359 +0000 UTC m=+5.667071192 container died 358aaa24df83b3f69dc7730cf6956c0796ff41b395cc21e7063313194bffb962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-11T12:36:54.275 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:54 smithi047 podman[101777]: 2024-05-11 12:36:54.119048805 +0000 UTC m=+6.211384636 container cleanup 358aaa24df83b3f69dc7730cf6956c0796ff41b395cc21e7063313194bffb962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-05-11T12:36:54.275 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:54 smithi047 bash[101777]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0 2024-05-11T12:36:54.775 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:54 smithi047 podman[101805]: 2024-05-11 12:36:54.413070396 +0000 UTC m=+0.835593281 container remove 358aaa24df83b3f69dc7730cf6956c0796ff41b395cc21e7063313194bffb962 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:36:55.125 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:54 smithi047 podman[101897]: 2024-05-11 12:36:54.761842354 +0000 UTC m=+0.018582058 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:36:55.125 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:54 smithi047 podman[101897]: 2024-05-11 12:36:54.880900656 +0000 UTC m=+0.137640375 container create add8e6badbd002912a5ca04427f34f2c9ccf1e1435b601f900817a1d1960ece4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-11T12:36:55.125 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:55 smithi047 podman[101897]: 2024-05-11 12:36:55.12544656 +0000 UTC m=+0.382186237 container init add8e6badbd002912a5ca04427f34f2c9ccf1e1435b601f900817a1d1960ece4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:36:55.525 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:55 smithi047 podman[101897]: 2024-05-11 12:36:55.129125004 +0000 UTC m=+0.385864688 container start add8e6badbd002912a5ca04427f34f2c9ccf1e1435b601f900817a1d1960ece4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-11T12:36:55.525 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:55 smithi047 podman[101897]: 2024-05-11 12:36:55.267376409 +0000 UTC m=+0.524116091 container attach add8e6badbd002912a5ca04427f34f2c9ccf1e1435b601f900817a1d1960ece4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-11T12:36:56.026 INFO:journalctl@ceph.osd.0.smithi047.stdout:May 11 12:36:55 smithi047 podman[101897]: 2024-05-11 12:36:55.690098301 +0000 UTC m=+0.946837982 container died add8e6badbd002912a5ca04427f34f2c9ccf1e1435b601f900817a1d1960ece4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-11T12:36:56.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.0.service' 2024-05-11T12:36:56.408 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:36:56.408 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-11T12:36:56.408 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-11T12:36:56.409 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.1 2024-05-11T12:36:56.702 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:36:56 smithi050 systemd[1]: Stopping Ceph osd.1 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:36:57.199 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:36:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1[38703]: 2024-05-11T12:36:56.730+0000 7f442b137700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:36:57.199 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:36:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1[38703]: 2024-05-11T12:36:56.730+0000 7f442b137700 -1 osd.1 86 *** Got signal Terminated *** 2024-05-11T12:36:57.200 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:36:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1[38703]: 2024-05-11T12:36:56.730+0000 7f442b137700 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:02.604 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:02 smithi050 podman[69186]: 2024-05-11 12:37:02.234023127 +0000 UTC m=+5.622991305 container died c1ba849aeec56069b04814ee6ef338d87d234b80efc887b0364cd919e6968046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-11T12:37:02.949 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:02 smithi050 podman[69186]: 2024-05-11 12:37:02.756583425 +0000 UTC m=+6.145551592 container cleanup c1ba849aeec56069b04814ee6ef338d87d234b80efc887b0364cd919e6968046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-11T12:37:02.949 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:02 smithi050 bash[69186]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1 2024-05-11T12:37:03.449 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:03 smithi050 podman[69204]: 2024-05-11 12:37:03.050820255 +0000 UTC m=+0.806952934 container remove c1ba849aeec56069b04814ee6ef338d87d234b80efc887b0364cd919e6968046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-11T12:37:03.845 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:03 smithi050 podman[69291]: 2024-05-11 12:37:03.44857253 +0000 UTC m=+0.018548993 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:37:03.846 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:03 smithi050 podman[69291]: 2024-05-11 12:37:03.593643241 +0000 UTC m=+0.163619753 container create 8c811e3726e037249931e9b9ad1e50e2aa839524688f009d16e62ba88860eb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:37:03.846 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:03 smithi050 podman[69291]: 2024-05-11 12:37:03.84603385 +0000 UTC m=+0.416010306 container init 8c811e3726e037249931e9b9ad1e50e2aa839524688f009d16e62ba88860eb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-05-11T12:37:04.199 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:03 smithi050 podman[69291]: 2024-05-11 12:37:03.849374251 +0000 UTC m=+0.419350709 container start 8c811e3726e037249931e9b9ad1e50e2aa839524688f009d16e62ba88860eb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True) 2024-05-11T12:37:04.199 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:03 smithi050 podman[69291]: 2024-05-11 12:37:03.946260519 +0000 UTC m=+0.516236986 container attach 8c811e3726e037249931e9b9ad1e50e2aa839524688f009d16e62ba88860eb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:37:04.765 INFO:journalctl@ceph.osd.1.smithi050.stdout:May 11 12:37:04 smithi050 podman[69291]: 2024-05-11 12:37:04.472088585 +0000 UTC m=+1.042065039 container died 8c811e3726e037249931e9b9ad1e50e2aa839524688f009d16e62ba88860eb0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:37:05.274 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.1.service' 2024-05-11T12:37:05.313 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:37:05.313 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-11T12:37:05.313 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-11T12:37:05.313 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.2 2024-05-11T12:37:05.614 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:05 smithi064 systemd[1]: Stopping Ceph osd.2 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:37:06.114 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2[39870]: 2024-05-11T12:37:05.624+0000 7f12ec373700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:37:06.114 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2[39870]: 2024-05-11T12:37:05.624+0000 7f12ec373700 -1 osd.2 86 *** Got signal Terminated *** 2024-05-11T12:37:06.114 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2[39870]: 2024-05-11T12:37:05.624+0000 7f12ec373700 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:11.334 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:11 smithi064 podman[70226]: 2024-05-11 12:37:11.022382984 +0000 UTC m=+5.512961905 container died f32bc7cbd729d6df990fcbd8f25e455405453f6cf764a90531071158f0d2e080 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-11T12:37:11.614 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:11 smithi064 podman[70226]: 2024-05-11 12:37:11.451174582 +0000 UTC m=+5.941753500 container cleanup f32bc7cbd729d6df990fcbd8f25e455405453f6cf764a90531071158f0d2e080 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, ceph=True) 2024-05-11T12:37:11.614 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:11 smithi064 bash[70226]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2 2024-05-11T12:37:12.114 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:11 smithi064 podman[70243]: 2024-05-11 12:37:11.687010865 +0000 UTC m=+0.666801598 container remove f32bc7cbd729d6df990fcbd8f25e455405453f6cf764a90531071158f0d2e080 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD) 2024-05-11T12:37:12.409 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:12 smithi064 podman[70333]: 2024-05-11 12:37:12.036465747 +0000 UTC m=+0.018650536 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:37:12.409 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:12 smithi064 podman[70333]: 2024-05-11 12:37:12.154566497 +0000 UTC m=+0.136751289 container create d92bb80ce19cf1d5df21933eb6cddbd1c517009f40ab5614aa1e93996818ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-11T12:37:12.410 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:12 smithi064 podman[70333]: 2024-05-11 12:37:12.409978948 +0000 UTC m=+0.392163740 container init d92bb80ce19cf1d5df21933eb6cddbd1c517009f40ab5614aa1e93996818ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux ) 2024-05-11T12:37:12.759 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:12 smithi064 podman[70333]: 2024-05-11 12:37:12.413301388 +0000 UTC m=+0.395486178 container start d92bb80ce19cf1d5df21933eb6cddbd1c517009f40ab5614aa1e93996818ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:37:12.759 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:12 smithi064 podman[70333]: 2024-05-11 12:37:12.510216434 +0000 UTC m=+0.492401231 container attach d92bb80ce19cf1d5df21933eb6cddbd1c517009f40ab5614aa1e93996818ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:37:13.047 INFO:journalctl@ceph.osd.2.smithi064.stdout:May 11 12:37:12 smithi064 podman[70333]: 2024-05-11 12:37:12.759874938 +0000 UTC m=+0.742059718 container died d92bb80ce19cf1d5df21933eb6cddbd1c517009f40ab5614aa1e93996818ed60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-11T12:37:13.372 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.2.service' 2024-05-11T12:37:13.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:37:13.413 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-11T12:37:13.413 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-11T12:37:13.414 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.3 2024-05-11T12:37:13.733 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:13 smithi047 systemd[1]: Stopping Ceph osd.3 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:37:14.025 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:13 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:37:13.733+0000 7f4fe38e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:37:14.025 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:13 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:37:13.733+0000 7f4fe38e2700 -1 osd.3 86 *** Got signal Terminated *** 2024-05-11T12:37:14.025 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:13 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:37:13.733+0000 7f4fe38e2700 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:14.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:14 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:14.670+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:14.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:14.511+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:14.956 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:14 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:14.593+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:15.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:14 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:14.957+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:15.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:15.343+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:15.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:15 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:15.576+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:15.775 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:15 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:37:15.521+0000 7f4fdfed6700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:50.510666+0000 front 2024-05-11T12:36:50.510705+0000 (oldest deadline 2024-05-11T12:37:14.610885+0000) 2024-05-11T12:37:15.826 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:15.550+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:16.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:15.719+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:16.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:15 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:15.927+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:16.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:16.348+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:16.775 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:16 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:37:16.502+0000 7f4fdfed6700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:50.510666+0000 front 2024-05-11T12:36:50.510705+0000 (oldest deadline 2024-05-11T12:37:14.610885+0000) 2024-05-11T12:37:16.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:16 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:16.592+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:16.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:16 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:16.542+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:17.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:16.716+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:17.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:16 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:16.902+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:17.596 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:17 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:37:17.520+0000 7f4fdfed6700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:50.510666+0000 front 2024-05-11T12:36:50.510705+0000 (oldest deadline 2024-05-11T12:37:14.610885+0000) 2024-05-11T12:37:17.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:17.345+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:17.909 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:17 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:17.596+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:17.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:17 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:17.590+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:18.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:17.725+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:18.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:17 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:17.910+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:18.759 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:18.378+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:18.921 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:18 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:18.580+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:18.921 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:18 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3[49598]: 2024-05-11T12:37:18.552+0000 7f4fdfed6700 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:50.510666+0000 front 2024-05-11T12:36:50.510705+0000 (oldest deadline 2024-05-11T12:37:14.610885+0000) 2024-05-11T12:37:18.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:18 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:18.578+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:19.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:18.758+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:19.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:18 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:18.921+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:19.557 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:19 smithi047 podman[102005]: 2024-05-11 12:37:19.280718863 +0000 UTC m=+5.659822562 container died 45d120627cc9d9680002041560ac957c5d7393e702d9556d3b5e81a747b5c680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-11T12:37:19.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:19.335+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:19.815 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:19.557+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:19.815 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:19 smithi047 podman[102005]: 2024-05-11 12:37:19.800042685 +0000 UTC m=+6.179146443 container cleanup 45d120627cc9d9680002041560ac957c5d7393e702d9556d3b5e81a747b5c680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-11T12:37:19.815 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:19 smithi047 bash[102005]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3 2024-05-11T12:37:19.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:19 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:19.588+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:20.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:19.774+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:20.275 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:20 smithi047 podman[102033]: 2024-05-11 12:37:20.05987839 +0000 UTC m=+0.782390640 container remove 45d120627cc9d9680002041560ac957c5d7393e702d9556d3b5e81a747b5c680 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-11T12:37:20.276 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:19 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:19.873+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:20.752 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:20.381+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:20.754 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:20 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:20.565+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:20.755 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:20 smithi047 podman[102122]: 2024-05-11 12:37:20.385097813 +0000 UTC m=+0.018765376 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:37:20.755 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:20 smithi047 podman[102122]: 2024-05-11 12:37:20.502812344 +0000 UTC m=+0.136479946 container create cd7d8851ac61af71eada38f3146ae2cd9dce8207ab32a1c72af3067ea861633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:37:20.755 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:20 smithi047 podman[102122]: 2024-05-11 12:37:20.755549713 +0000 UTC m=+0.389217279 container init cd7d8851ac61af71eada38f3146ae2cd9dce8207ab32a1c72af3067ea861633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-11T12:37:20.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:20 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:20.629+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:21.025 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:20 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:20.911+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:21.025 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:20 smithi047 podman[102122]: 2024-05-11 12:37:20.759191847 +0000 UTC m=+0.392859410 container start cd7d8851ac61af71eada38f3146ae2cd9dce8207ab32a1c72af3067ea861633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-11T12:37:21.025 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:20 smithi047 podman[102122]: 2024-05-11 12:37:20.847221996 +0000 UTC m=+0.480889572 container attach cd7d8851ac61af71eada38f3146ae2cd9dce8207ab32a1c72af3067ea861633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:37:21.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:20.751+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:21.383 INFO:journalctl@ceph.osd.3.smithi047.stdout:May 11 12:37:21 smithi047 podman[102122]: 2024-05-11 12:37:21.068086842 +0000 UTC m=+0.701754406 container died cd7d8851ac61af71eada38f3146ae2cd9dce8207ab32a1c72af3067ea861633e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-11T12:37:21.758 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:21.370+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:21.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:21 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:21.519+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:21.822 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.3.service' 2024-05-11T12:37:21.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:37:21.869 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-11T12:37:21.869 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-11T12:37:21.870 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.4 2024-05-11T12:37:21.896 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:21 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:21.613+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:22.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:21.757+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:22.199 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:21 smithi050 systemd[1]: Stopping Ceph osd.4 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:37:22.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:21 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:21.865+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:22.651 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:22 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4[45670]: 2024-05-11T12:37:22.311+0000 7f39da8a3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:37:22.652 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:22 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4[45670]: 2024-05-11T12:37:22.311+0000 7f39da8a3700 -1 osd.4 86 *** Got signal Terminated *** 2024-05-11T12:37:22.652 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:22 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4[45670]: 2024-05-11T12:37:22.311+0000 7f39da8a3700 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:22.712 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:22.382+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:22.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:22 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:22.493+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:22.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:22 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:22.651+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:22.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:22 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:22.651+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:23.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:22.711+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:23.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:22 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:22.864+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:23.694 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:23.398+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:23.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:23 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:23.519+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:23.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:23 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:23.519+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:23.887 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:23 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:23.629+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:23.887 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:23 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:23.629+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:24.096 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:23.693+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:24.096 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:23.693+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:24.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:23 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:23.887+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:24.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:23 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:23.895+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:24.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:23 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:23.895+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:24.688 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:24 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:24.386+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:24.688 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:24 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:24.386+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:24.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:24 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:24.503+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:24.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:24 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:24.503+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:24.891 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:24 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:24.625+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:24.891 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:24 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:24.625+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:25.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:24 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:24.697+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:25.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:24 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:24.697+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:25.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:24 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:24.890+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:25.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:24 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:24.936+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:25.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:24 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:24.936+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:25.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:25 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:25.344+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:25.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:25 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:25.344+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:25.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:25 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:25.514+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:25.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:25 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:25.514+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:25.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:25 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:25.864+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:25.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:25 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:25.657+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:25.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:25 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:25.657+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:26.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:25 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:25.721+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:26.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:25 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:25.721+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:26.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:25 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:25.956+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:26.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:25 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:25.956+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:26.449 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:25 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4[45670]: 2024-05-11T12:37:25.985+0000 7f39d2e8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.593189+0000 front 2024-05-11T12:36:59.593101+0000 (oldest deadline 2024-05-11T12:37:25.493246+0000) 2024-05-11T12:37:26.716 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:26 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:26.374+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:26.716 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:26 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:26.374+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:26.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:26 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:26.473+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:26.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:26 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:26.473+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:26.949 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:26 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4[45670]: 2024-05-11T12:37:26.944+0000 7f39d2e8f700 -1 osd.4 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.593189+0000 front 2024-05-11T12:36:59.593101+0000 (oldest deadline 2024-05-11T12:37:25.493246+0000) 2024-05-11T12:37:26.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:26 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:26.862+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:26.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:26 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:26.696+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:26.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:26 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:26.696+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:27.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:26 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:26.715+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:27.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:26 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:26.715+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:27.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:26 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:26.919+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:27.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:26 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:26.919+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:27.375 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:27 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:27.118+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:27.698 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:27 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:27.373+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:27.698 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:27 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:27.373+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:27.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:27 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:27.501+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:27.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:27 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:27.501+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:28.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:27 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:27.698+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:28.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:27 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:27.698+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:28.175 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:27 smithi050 podman[69409]: 2024-05-11 12:37:27.730105119 +0000 UTC m=+5.664175534 container died e327c0773da944fd8086d97f584043d9cd455a2a44bd1c005294a39be486dd64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-11T12:37:28.176 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:27 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:27.721+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:28.176 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:27 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:27.721+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:28.176 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:27 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:27.852+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:28.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:27 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:27.918+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:28.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:27 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:27.918+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:28.401 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:28 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:28.139+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:28.449 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:28 smithi050 podman[69409]: 2024-05-11 12:37:28.270704909 +0000 UTC m=+6.204775317 container cleanup e327c0773da944fd8086d97f584043d9cd455a2a44bd1c005294a39be486dd64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-05-11T12:37:28.449 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:28 smithi050 bash[69409]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4 2024-05-11T12:37:28.677 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:28 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:28.400+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:28.677 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:28 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:28.400+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:28.825 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:28 smithi050 podman[69428]: 2024-05-11 12:37:28.505468071 +0000 UTC m=+0.768787714 container remove e327c0773da944fd8086d97f584043d9cd455a2a44bd1c005294a39be486dd64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-11T12:37:28.825 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:28.712+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:28.825 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:28.712+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:28.885 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:28 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:28.533+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:28.885 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:28 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:28.533+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:29.097 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:28 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:28.824+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:29.098 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:28 smithi050 podman[69521]: 2024-05-11 12:37:28.848154384 +0000 UTC m=+0.018583915 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:37:29.098 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:28 smithi050 podman[69521]: 2024-05-11 12:37:28.973410355 +0000 UTC m=+0.143839873 container create 18ecdb5613e41ceb63827cd3bae459c5eacf1b560fd333d199eb3b28ee64eaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:37:29.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:28 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:28.676+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:29.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:28 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:28.676+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:29.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:28 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:28.884+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:29.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:28 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:28.884+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:29.449 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:29 smithi050 podman[69521]: 2024-05-11 12:37:29.210408864 +0000 UTC m=+0.380838383 container init 18ecdb5613e41ceb63827cd3bae459c5eacf1b560fd333d199eb3b28ee64eaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:37:29.449 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:29 smithi050 podman[69521]: 2024-05-11 12:37:29.213767332 +0000 UTC m=+0.384196853 container start 18ecdb5613e41ceb63827cd3bae459c5eacf1b560fd333d199eb3b28ee64eaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:37:29.449 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:29 smithi050 podman[69521]: 2024-05-11 12:37:29.309509221 +0000 UTC m=+0.479938752 container attach 18ecdb5613e41ceb63827cd3bae459c5eacf1b560fd333d199eb3b28ee64eaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-11T12:37:29.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:29 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:29.184+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:29.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:29 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:29.433+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:29.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:29 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:29.433+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:29.803 INFO:journalctl@ceph.osd.4.smithi050.stdout:May 11 12:37:29 smithi050 podman[69521]: 2024-05-11 12:37:29.517812596 +0000 UTC m=+0.688242115 container died 18ecdb5613e41ceb63827cd3bae459c5eacf1b560fd333d199eb3b28ee64eaf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:37:29.803 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:29 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:29.712+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:29.803 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:29 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:29.712+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:29.878 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:29 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:29.560+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:29.878 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:29 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:29.560+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:30.083 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:29 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:29.802+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:30.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:29 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:29.649+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:30.114 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:29 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:29.649+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:30.138 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.4.service' 2024-05-11T12:37:30.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:37:30.222 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-11T12:37:30.222 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-11T12:37:30.222 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.5 2024-05-11T12:37:30.278 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:29 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:29.877+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:30.278 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:29 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:29.877+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:30.560 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:30.232+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:30.560 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:30.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:30.560 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:30.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:30.560 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:30 smithi064 systemd[1]: Stopping Ceph osd.5 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:37:30.561 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:30.558+0000 7f88573a6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:37:30.561 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:30.558+0000 7f88573a6700 -1 osd.5 86 *** Got signal Terminated *** 2024-05-11T12:37:30.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:30.558+0000 7f88573a6700 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:30.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:30.634+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:30.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:30 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:30.634+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:30.924 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:30 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:30.557+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:30.924 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:30 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:30.557+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:31.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:30 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:30.789+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:31.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:30 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:30.741+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:31.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:30 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:30.741+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:31.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:30 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:30.924+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:31.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:30 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:30.924+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:31.598 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:31 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:31.246+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:31.598 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:31 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:31.466+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:31.598 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:31 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:31.466+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:31.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:31 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:31.598+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:31.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:31 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:31.598+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:31.897 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:31 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:31.534+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:31.897 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:31 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:31.534+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:31.897 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:31 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:31.534+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:32.100 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:31 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:31.744+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:32.101 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:31 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:31.744+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:32.101 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:31 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:31.820+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:32.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:31 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:31.897+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:32.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:31 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:31.897+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:32.606 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:32 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:32.229+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:32.606 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:32 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:32.230+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:32.606 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:32 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:32.444+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:32.606 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:32 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:32.444+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:32.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:32 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:32.605+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:32.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:32 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:32.605+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:32.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:32 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:32.605+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.659515+0000 front 2024-05-11T12:37:06.659491+0000 (oldest deadline 2024-05-11T12:37:31.959596+0000) 2024-05-11T12:37:32.894 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:32 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:32.569+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:32.894 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:32 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:32.569+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:32.894 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:32 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:32.569+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:33.112 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:32 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:32.813+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:33.112 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:32 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:32.813+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:33.113 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:32 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:32.714+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:33.113 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:32 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:32.714+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:33.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:32 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:32.894+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:33.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:32 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:32.894+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:33.488 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:33 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:33.193+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:33.488 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:33 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:33.193+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:33.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:33 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:33.560+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:33.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:33 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:33.560+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:33.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:33 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:33.560+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.659515+0000 front 2024-05-11T12:37:06.659491+0000 (oldest deadline 2024-05-11T12:37:31.959596+0000) 2024-05-11T12:37:33.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:33 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:33.488+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:33.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:33 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:33.488+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:33.884 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:33.536+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:33.884 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:33.536+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:33.884 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:33.536+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:34.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:33 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:33.844+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:34.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:33 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:33.844+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:34.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:33 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:33.705+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:34.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:33 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:33.705+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:34.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:33.884+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:34.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:33 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:33.884+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:34.472 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:34 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:34.178+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:34.472 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:34 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:34.178+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:34.861 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:34.566+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:34.861 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:34.566+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:34.861 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:34.566+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:34.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:34 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:34.513+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:34.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:34 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:34.513+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:34.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:34 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:34.513+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.659515+0000 front 2024-05-11T12:37:06.659491+0000 (oldest deadline 2024-05-11T12:37:31.959596+0000) 2024-05-11T12:37:34.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:34 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:34.471+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:34.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:34 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:34.471+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:35.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:34 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:34.879+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:35.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:34 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:34.879+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:35.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:34 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:34.751+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:35.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:34 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:34.751+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:35.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:34 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:34.751+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:35.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:34.861+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:35.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:34 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:34.861+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:35.479 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:35.146+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:35.479 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:35.146+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:35.858 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:35 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:35.587+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:35.859 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:35 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:35.587+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:35.859 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:35 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:35.587+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:35.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:35.503+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.156471+0000 front 2024-05-11T12:36:52.156301+0000 (oldest deadline 2024-05-11T12:37:14.456288+0000) 2024-05-11T12:37:35.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:35.503+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:57.858310+0000 front 2024-05-11T12:36:57.858382+0000 (oldest deadline 2024-05-11T12:37:23.158324+0000) 2024-05-11T12:37:35.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5[46411]: 2024-05-11T12:37:35.503+0000 7f885399a700 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.659515+0000 front 2024-05-11T12:37:06.659491+0000 (oldest deadline 2024-05-11T12:37:31.959596+0000) 2024-05-11T12:37:35.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:35.479+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:35.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:35.479+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:35.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:35 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:35.479+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:36.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:35 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:35.850+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:36.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:35 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:35.850+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:36.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:35 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:35.735+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:36.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:35 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:35.735+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:36.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:35 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:35.735+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:36.235 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:36 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:36.159+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:36.236 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:36 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:36.159+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:36.236 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:35 smithi064 podman[70451]: 2024-05-11 12:37:35.958195474 +0000 UTC m=+5.538399338 container died 0c6bcc833beac84d1364f6af202c36f5fb5673e74cbafc844d838a4b30afd91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-11T12:37:36.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:35 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:35.858+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:36.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:35 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:35.858+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:36.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:36 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:36.477+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:36.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:36 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:36.477+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:36.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:36 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:36.477+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:36.615 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:36 smithi064 podman[70451]: 2024-05-11 12:37:36.335212971 +0000 UTC m=+5.915416831 container cleanup 0c6bcc833beac84d1364f6af202c36f5fb5673e74cbafc844d838a4b30afd91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-11T12:37:36.615 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:36 smithi064 bash[70451]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5 2024-05-11T12:37:36.902 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:36.565+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:36.903 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:36.565+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:36.903 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:36.565+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:36.966 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:36 smithi064 podman[70468]: 2024-05-11 12:37:36.61988234 +0000 UTC m=+0.663368724 container remove 0c6bcc833beac84d1364f6af202c36f5fb5673e74cbafc844d838a4b30afd91a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-11T12:37:37.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:36 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:36.839+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:37.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:36 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:36.839+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:37.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:36 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:36.764+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:37.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:36 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:36.764+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:37.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:36 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:36.764+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:37.265 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:37 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:37.159+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:37.265 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:37 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:37.159+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:37.265 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:36 smithi064 podman[70559]: 2024-05-11 12:37:36.869818556 +0000 UTC m=+0.018716505 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:37:37.265 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:36 smithi064 podman[70559]: 2024-05-11 12:37:36.996112283 +0000 UTC m=+0.145010288 container create fa07023accb69c94932cb4d447e2a6aee1cb57154808867a55e381dbdc422906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:37:37.265 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:37 smithi064 podman[70559]: 2024-05-11 12:37:37.265588182 +0000 UTC m=+0.414486124 container init fa07023accb69c94932cb4d447e2a6aee1cb57154808867a55e381dbdc422906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:37:37.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:36.901+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:37.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:36.901+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:37.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:36 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:36.901+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:04.806372+0000 front 2024-05-11T12:37:04.806223+0000 (oldest deadline 2024-05-11T12:37:36.006490+0000) 2024-05-11T12:37:37.580 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:37 smithi064 podman[70559]: 2024-05-11 12:37:37.268959855 +0000 UTC m=+0.417857794 container start fa07023accb69c94932cb4d447e2a6aee1cb57154808867a55e381dbdc422906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-11T12:37:37.580 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:37 smithi064 podman[70559]: 2024-05-11 12:37:37.348899293 +0000 UTC m=+0.497797230 container attach fa07023accb69c94932cb4d447e2a6aee1cb57154808867a55e381dbdc422906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=) 2024-05-11T12:37:37.580 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:37 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:37.445+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:37.580 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:37 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:37.445+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:37.580 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:37 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:37.445+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:37.864 INFO:journalctl@ceph.osd.5.smithi064.stdout:May 11 12:37:37 smithi064 podman[70559]: 2024-05-11 12:37:37.580318691 +0000 UTC m=+0.729216632 container died fa07023accb69c94932cb4d447e2a6aee1cb57154808867a55e381dbdc422906 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-11T12:37:37.929 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:37.575+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:37.929 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:37.575+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:37.929 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:37.575+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:38.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:37 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:37.835+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:38.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:37 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:37.835+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:38.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:37 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:37.767+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:38.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:37 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:37.767+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:38.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:37 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:37.767+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:38.209 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.5.service' 2024-05-11T12:37:38.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:37:38.249 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-11T12:37:38.249 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-11T12:37:38.249 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.6 2024-05-11T12:37:38.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:37.929+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:38.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:37.929+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:38.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:37.929+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:04.806372+0000 front 2024-05-11T12:37:04.806223+0000 (oldest deadline 2024-05-11T12:37:36.006490+0000) 2024-05-11T12:37:38.461 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:38 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:38.205+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:38.461 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:38 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:38.205+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:38.540 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:38 smithi047 systemd[1]: Stopping Ceph osd.6 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:37:38.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:38 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:38.460+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:38.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:38 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:38.460+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:38.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:38 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:38.460+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:38.893 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:38.540+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:38.540+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:38.540+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:38.561+0000 7f5998451700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:38.561+0000 7f5998451700 -1 osd.6 86 *** Got signal Terminated *** 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:38.561+0000 7f5998451700 -1 osd.6 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:38.893+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:38.893+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:38.894 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:38 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:38.893+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:04.806372+0000 front 2024-05-11T12:37:04.806223+0000 (oldest deadline 2024-05-11T12:37:36.006490+0000) 2024-05-11T12:37:39.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:38 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:38.849+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:39.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:38 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:38.849+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:39.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:38 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:38.753+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:39.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:38 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:38.753+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:39.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:38 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:38.753+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:39.509 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:39 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:39.225+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:39.510 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:39 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:39.225+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:39.845 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:39.542+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:39.845 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:39.542+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:39.845 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:39.542+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:39.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:39 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:39.509+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:39.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:39 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:39.509+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:39.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:39 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:39.509+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:40.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:39 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:39.837+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:40.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:39 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:39.837+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:40.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:39 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:39.790+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:40.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:39 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:39.790+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:40.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:39 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:39.790+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:40.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:39.844+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:40.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:39.844+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:40.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:39 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:39.844+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:04.806372+0000 front 2024-05-11T12:37:04.806223+0000 (oldest deadline 2024-05-11T12:37:36.006490+0000) 2024-05-11T12:37:40.527 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:40 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:40.226+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:40.527 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:40 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:40.226+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:40.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:40 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:40.527+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:40.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:40 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:40.527+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:40.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:40 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:40.527+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:40.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:40 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:40.527+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:41.025 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:40.816+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:41.025 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:40.816+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:41.026 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:40.816+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:04.806372+0000 front 2024-05-11T12:37:04.806223+0000 (oldest deadline 2024-05-11T12:37:36.006490+0000) 2024-05-11T12:37:41.026 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:40.576+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:41.026 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:40.576+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:41.026 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:40.576+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:41.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:40 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:40.798+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:41.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:40 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:40.798+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:41.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:40 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:40.836+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:41.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:40 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:40.836+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:41.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:40 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:40.836+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:41.506 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:41 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:41.247+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:41.506 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:41 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:41.247+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:41.800 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:41 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:41.506+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:41.800 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:41 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:41.506+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:41.800 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:41 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:41.506+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:41.801 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:41 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:41.506+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:41.852 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:41.528+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:41.852 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:41.528+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:41.852 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:41.528+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:42.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:41 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:41.838+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:42.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:41 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:41.838+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:42.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:41 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:41.818+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:42.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:41 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:41.818+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:42.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:41 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:41.818+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:42.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:41.852+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:42.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:41.852+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:42.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:41.852+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:04.806372+0000 front 2024-05-11T12:37:04.806223+0000 (oldest deadline 2024-05-11T12:37:36.006490+0000) 2024-05-11T12:37:42.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:41 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:41.852+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.006873+0000 front 2024-05-11T12:37:16.006920+0000 (oldest deadline 2024-05-11T12:37:41.306881+0000) 2024-05-11T12:37:42.478 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:42.225+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:42.478 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:42.225+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:42.774 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:42.478+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:42.775 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:42.478+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:42.775 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:42.478+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:42.775 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:42 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:42.478+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:42.899 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:42.542+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:42.899 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:42.542+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:42.899 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:42.542+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:42.899 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:42.542+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:43.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:42 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:42.820+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:43.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:42 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:42.820+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:43.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:42 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:42.820+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:43.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:42 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:42.783+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:43.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:42 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:42.783+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:43.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:42 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:42.783+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:43.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:42 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:42.783+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:43.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:42.898+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.603637+0000 front 2024-05-11T12:36:49.603593+0000 (oldest deadline 2024-05-11T12:37:14.903815+0000) 2024-05-11T12:37:43.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:42.898+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.605228+0000 front 2024-05-11T12:36:59.605198+0000 (oldest deadline 2024-05-11T12:37:23.705071+0000) 2024-05-11T12:37:43.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:42.898+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:04.806372+0000 front 2024-05-11T12:37:04.806223+0000 (oldest deadline 2024-05-11T12:37:36.006490+0000) 2024-05-11T12:37:43.275 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:42 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6[56584]: 2024-05-11T12:37:42.898+0000 7f5990a3d700 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.006873+0000 front 2024-05-11T12:37:16.006920+0000 (oldest deadline 2024-05-11T12:37:41.306881+0000) 2024-05-11T12:37:43.474 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:43 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:43.216+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:43.475 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:43 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:43.216+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:43.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:43 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:43.513+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:43.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:43 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:43.513+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:43.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:43 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:43.513+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:43.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:43 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:43.513+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:43.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:43 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:43.474+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:43.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:43 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:43.474+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:43.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:43 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:43.474+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:43.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:43 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:43.474+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:44.202 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:43.857+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:44.202 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:43.857+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:44.202 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:43.857+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:44.202 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:43.743+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:44.203 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:43.743+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:44.203 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:43.743+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:44.203 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:43 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:43.743+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:44.263 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:43 smithi047 podman[102227]: 2024-05-11 12:37:43.968742771 +0000 UTC m=+5.513824708 container died 709c72be3619600eec876e382f4ed492857226a0cbada1b40870dfcf7ed0fa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-05-11T12:37:44.525 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:44 smithi047 podman[102227]: 2024-05-11 12:37:44.4632254 +0000 UTC m=+6.008307368 container cleanup 709c72be3619600eec876e382f4ed492857226a0cbada1b40870dfcf7ed0fa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-11T12:37:44.525 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:44 smithi047 bash[102227]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6 2024-05-11T12:37:44.526 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:44 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:44.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:44.526 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:44 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:44.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:44.526 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:44 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:44.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:44.526 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:44 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:44.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:44.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:44 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:44.220+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:44.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:44 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:44.220+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:44.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:44 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:44.220+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:44.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:44 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:44.449+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:44.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:44 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:44.449+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:44.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:44 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:44.449+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:44.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:44 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:44.449+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:45.025 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:44 smithi047 podman[102245]: 2024-05-11 12:37:44.722982893 +0000 UTC m=+0.753826549 container remove 709c72be3619600eec876e382f4ed492857226a0cbada1b40870dfcf7ed0fa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-11T12:37:45.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:44 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:44.859+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:45.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:44 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:44.859+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:45.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:44 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:44.859+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:45.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:44 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:44.726+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:45.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:44 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:44.726+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:45.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:44 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:44.726+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:45.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:44 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:44.726+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:45.451 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:45 smithi047 podman[102334]: 2024-05-11 12:37:45.054278257 +0000 UTC m=+0.018852663 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:37:45.451 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:45 smithi047 podman[102334]: 2024-05-11 12:37:45.173864136 +0000 UTC m=+0.138438523 container create 9639465215f5374a24d88dfbe64c38e16eaeee4303a52ef059bed1589fa37384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-11T12:37:45.458 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:45 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:45.184+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:45.458 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:45 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:45.184+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:45.458 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:45 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:45.184+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:45.764 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:45 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:45.471+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:45.765 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:45 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:45.471+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:45.765 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:45 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:45.471+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:45.765 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:45 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:45.471+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:45.765 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:45 smithi047 podman[102334]: 2024-05-11 12:37:45.451571629 +0000 UTC m=+0.416146029 container init 9639465215f5374a24d88dfbe64c38e16eaeee4303a52ef059bed1589fa37384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-11T12:37:45.765 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:45 smithi047 podman[102334]: 2024-05-11 12:37:45.455296707 +0000 UTC m=+0.419871095 container start 9639465215f5374a24d88dfbe64c38e16eaeee4303a52ef059bed1589fa37384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-11T12:37:45.765 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:45 smithi047 podman[102334]: 2024-05-11 12:37:45.543294479 +0000 UTC m=+0.507868898 container attach 9639465215f5374a24d88dfbe64c38e16eaeee4303a52ef059bed1589fa37384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:37:45.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:45 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:45.457+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:45.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:45 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:45.457+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:45.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:45 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:45.457+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:45.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:45 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:45.457+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:46.025 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:45 smithi047 podman[102334]: 2024-05-11 12:37:45.765227454 +0000 UTC m=+0.729801848 container died 9639465215f5374a24d88dfbe64c38e16eaeee4303a52ef059bed1589fa37384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-05-11T12:37:46.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:45.828+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:46.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:45.828+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:46.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:45.828+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:46.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:45.766+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:46.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:45.766+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:46.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:45.766+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:46.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:45 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:45.766+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:46.403 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.6.service' 2024-05-11T12:37:46.469 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:46 smithi047 podman[102360]: 2024-05-11 12:37:46.380389967 +0000 UTC m=+0.605781653 container remove 9639465215f5374a24d88dfbe64c38e16eaeee4303a52ef059bed1589fa37384 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:37:46.469 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:46 smithi047 systemd[1]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.6.service: Deactivated successfully. 2024-05-11T12:37:46.470 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:46 smithi047 systemd[1]: Stopped Ceph osd.6 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:37:46.470 INFO:journalctl@ceph.osd.6.smithi047.stdout:May 11 12:37:46 smithi047 systemd[1]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.6.service: Consumed 8min 50.667s CPU time. 2024-05-11T12:37:46.486 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:37:46.487 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-11T12:37:46.487 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-11T12:37:46.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.7 2024-05-11T12:37:46.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:46.231+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:46.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:46.231+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:46.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:46.231+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:46.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:46.472+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:46.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:46.472+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:46.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:46.472+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:46.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:46 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:46.472+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:46.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:46 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:46.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:46.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:46 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:46.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:46.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:46 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:46.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:46.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:46 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:46.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:46.780 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:46 smithi050 systemd[1]: Stopping Ceph osd.7 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:37:47.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:46.832+0000 7f06cb697700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:37:47.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:46.832+0000 7f06cb697700 -1 osd.7 86 *** Got signal Terminated *** 2024-05-11T12:37:47.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:46.832+0000 7f06cb697700 -1 osd.7 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:47.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:46.852+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:47.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:46.852+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:47.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:46.852+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:47.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:46.779+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:47.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:46.779+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:47.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:46.779+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:47.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:46.779+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:47.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:47 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:47.260+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:47.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:47 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:47.260+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:47.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:47 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:47.260+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:47.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:47 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:47.429+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:47.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:47 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:47.429+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:47.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:47 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:47.429+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:47.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:47 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:47.429+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:47.652 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:47 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:47.524+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:47.652 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:47 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:47.524+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:47.652 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:47 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:47.524+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:47.652 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:47 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:47.524+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:48.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:47 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:47.857+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:48.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:47 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:47.857+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:48.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:47 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:47.857+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:48.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:47 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:47.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:48.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:47 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:47.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:48.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:47 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:47.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:48.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:47 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:47.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:48.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:48 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:48.282+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:48.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:48 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:48.282+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:48.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:48 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:48.282+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:48.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:48 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:48.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:48.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:48 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:48.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:48.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:48 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:48.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:48.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:48 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:48.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:49.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:48.529+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:49.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:48.529+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:49.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:48.529+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:49.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:48.529+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:49.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:48 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:48.529+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:49.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:48 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:48.848+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:49.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:48 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:48.848+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:49.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:48 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:48.848+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:49.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:48 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:48.733+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:49.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:48 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:48.733+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:49.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:48 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:48.733+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:49.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:48 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:48.733+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:49.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:49.292+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:49.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:49.292+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:49.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:49.292+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:49.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:49.292+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:49.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:49.505+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:49.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:49.505+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:49.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:49.505+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:49.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:49.505+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:49.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:49 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:49.505+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:50.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:49 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:49.549+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:50.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:49 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:49.549+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:50.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:49 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:49.549+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:50.026 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:49 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:49.549+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:50.026 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:49 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:49.549+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:50.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:49 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:49.807+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:50.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:49 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:49.807+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:50.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:49 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:49.807+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:50.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:49 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:49.732+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:50.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:49 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:49.732+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:50.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:49 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:49.732+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:50.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:49 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:49.732+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:50.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:50.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:50.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:50.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:50.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:50.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:50.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:50.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:50.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:50.535+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:50.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:50.535+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:50.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:50.535+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:50.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:50.535+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:50.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:50 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:50.535+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:50.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:50 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:50.517+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:50.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:50 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:50.517+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:50.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:50 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:50.517+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:50.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:50 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:50.517+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:50.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:50 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:50.517+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:51.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:50.804+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:51.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:50.804+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:51.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:50.804+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:51.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:50.804+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:26.941443+0000 front 2024-05-11T12:37:26.941466+0000 (oldest deadline 2024-05-11T12:37:49.841528+0000) 2024-05-11T12:37:51.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:50.776+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:51.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:50.776+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:51.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:50.776+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:51.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:50 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:50.776+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:51.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:51.353+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:51.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:51.353+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:51.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:51.353+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:51.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:51.353+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:51.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:51.544+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:51.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:51.544+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:51.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:51.544+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:51.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:51.544+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:51.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:51 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:51.544+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:51.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:51 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:51.520+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:51.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:51 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:51.520+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:51.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:51 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:51.520+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:51.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:51 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:51.520+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:51.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:51 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:51.520+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:52.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:51.831+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:36:59.637261+0000 front 2024-05-11T12:36:59.636915+0000 (oldest deadline 2024-05-11T12:37:23.136951+0000) 2024-05-11T12:37:52.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:51.831+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.038162+0000 front 2024-05-11T12:37:06.038085+0000 (oldest deadline 2024-05-11T12:37:31.938135+0000) 2024-05-11T12:37:52.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:51.831+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:13.639057+0000 front 2024-05-11T12:37:13.639138+0000 (oldest deadline 2024-05-11T12:37:41.839733+0000) 2024-05-11T12:37:52.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7[52514]: 2024-05-11T12:37:51.831+0000 7f06c3c83700 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:26.941443+0000 front 2024-05-11T12:37:26.941466+0000 (oldest deadline 2024-05-11T12:37:49.841528+0000) 2024-05-11T12:37:52.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:51.806+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:52.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:51.806+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:52.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:51.806+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:52.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:51.806+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:52.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:51.806+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:52.520 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:52.379+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:52.520 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:52.379+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:52.520 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:52.379+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:52.520 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:52.379+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:52.676 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:52 smithi050 podman[69642]: 2024-05-11 12:37:52.393955209 +0000 UTC m=+5.705238618 container died 3fefd492c163333231da1516e48bf25fbab6fd057a5d0b12d5c0b15897e13f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-11T12:37:52.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:52 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:52.486+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:52.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:52 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:52.486+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:52.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:52 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:52.486+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:52.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:52 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:52.486+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:52.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:52 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:52.486+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:52.850 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:52.520+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:52.850 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:52.520+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:52.850 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:52.520+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:52.850 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:52.520+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:52.850 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:52 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:52.520+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:52.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:52 smithi050 podman[69642]: 2024-05-11 12:37:52.784013215 +0000 UTC m=+6.095296624 container cleanup 3fefd492c163333231da1516e48bf25fbab6fd057a5d0b12d5c0b15897e13f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-05-11T12:37:52.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:52 smithi050 bash[69642]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7 2024-05-11T12:37:52.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:52 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:52.819+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:52.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:52 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:52.819+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:52.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:52 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:52.819+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:52.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:52 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:52.819+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:52.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:52 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:52.819+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:53.388 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:53 smithi050 podman[69658]: 2024-05-11 12:37:53.06902963 +0000 UTC m=+0.670558823 container remove 3fefd492c163333231da1516e48bf25fbab6fd057a5d0b12d5c0b15897e13f87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-11T12:37:53.639 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:53 smithi050 podman[69749]: 2024-05-11 12:37:53.292073383 +0000 UTC m=+0.018376028 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:37:53.639 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:53 smithi050 podman[69749]: 2024-05-11 12:37:53.420268729 +0000 UTC m=+0.146571409 container create a4958ac044ed62804953ee0c3000b8b5c9d42b54dd9bf558530ba106fd706845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True) 2024-05-11T12:37:53.639 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:53 smithi050 podman[69749]: 2024-05-11 12:37:53.639499073 +0000 UTC m=+0.365801692 container init a4958ac044ed62804953ee0c3000b8b5c9d42b54dd9bf558530ba106fd706845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-05-11T12:37:53.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:53.510+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:53.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:53.510+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:53.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:53.510+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:53.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:53.510+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:53.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:53 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:53.510+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:53.864 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:53.377+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:53.864 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:53.377+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:53.864 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:53.377+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:53.864 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:53.377+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:53.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:53.502+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:53.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:53.502+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:53.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:53.502+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:53.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:53.502+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:53.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:53 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:53.502+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:53.945 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:53 smithi050 podman[69749]: 2024-05-11 12:37:53.643040853 +0000 UTC m=+0.369343472 container start a4958ac044ed62804953ee0c3000b8b5c9d42b54dd9bf558530ba106fd706845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-11T12:37:53.945 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:53 smithi050 podman[69749]: 2024-05-11 12:37:53.781254427 +0000 UTC m=+0.507557058 container attach a4958ac044ed62804953ee0c3000b8b5c9d42b54dd9bf558530ba106fd706845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-11T12:37:53.945 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:53 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:53.800+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:53.946 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:53 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:53.800+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:53.946 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:53 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:53.800+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:53.946 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:53 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:53.800+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:53.946 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:53 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:53.800+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:54.199 INFO:journalctl@ceph.osd.7.smithi050.stdout:May 11 12:37:53 smithi050 podman[69749]: 2024-05-11 12:37:53.946004309 +0000 UTC m=+0.672306941 container died a4958ac044ed62804953ee0c3000b8b5c9d42b54dd9bf558530ba106fd706845 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-11T12:37:54.592 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.7.service' 2024-05-11T12:37:54.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:54.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:54.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:54.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:54.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:54.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:54.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:54.338+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:54.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:54.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:54.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:54.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:54.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:54.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:54.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:54.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:54.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:54.464+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:54.634 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:37:54.635 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-11T12:37:54.635 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-11T12:37:54.635 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.8 2024-05-11T12:37:54.865 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:54 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:54.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:54.865 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:54 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:54.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:54.865 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:54 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:54.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:54.865 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:54 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:54.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:54.865 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:54 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:54.763+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:54.956 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 systemd[1]: Stopping Ceph osd.8 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:37:54.957 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:54.956+0000 7f7348553700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:37:54.957 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:54.956+0000 7f7348553700 -1 osd.8 86 *** Got signal Terminated *** 2024-05-11T12:37:54.957 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:54 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:54.956+0000 7f7348553700 -1 osd.8 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:37:55.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:54 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:54.560+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:55.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:54 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:54.560+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:55.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:54 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:54.560+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:55.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:54 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:54.560+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:55.025 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:54 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:54.560+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:55.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:55.358+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:55.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:55.358+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:55.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:55.358+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:55.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:55.358+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:55.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:55.485+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:55.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:55.485+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:55.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:55.485+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:55.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:55.485+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:55.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:55 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:55.485+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:55.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:55.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:55.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:55.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:55.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:55.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:55.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:55.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:55.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:55 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:55.512+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:56.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:55 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:55.811+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:56.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:55 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:55.811+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:56.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:55 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:55.811+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:56.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:55 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:55.811+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:56.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:55 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:55.811+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:56.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:56.345+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:56.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:56.345+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:56.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:56.345+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:56.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:56.345+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:56.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:56.486+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:56.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:56.486+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:56.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:56.486+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:56.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:56.486+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:56.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:56 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:56.486+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:56.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:56.507+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:56.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:56.507+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:56.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:56.507+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:56.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:56.507+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:56.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:56 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:56.507+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:57.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:56.842+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:57.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:56.842+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:57.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:56.842+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:57.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:56.842+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:57.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:56 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:56.842+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:57.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:57.354+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:57.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:57.354+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:57.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:57.354+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:57.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:57.354+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:57.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:57.533+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:57.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:57.533+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:57.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:57.533+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:57.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:57.533+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:57.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:57.533+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:57.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:57 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:57.533+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:37:57.740 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:57.461+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:57.740 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:57.461+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:57.741 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:57.461+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:57.741 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:57.461+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:57.741 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:57 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:57.461+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:58.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:57.851+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:58.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:57.851+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:58.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:57.851+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:58.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:57.851+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:58.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:57.851+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:58.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:57.851+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:37:58.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:58.315+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:58.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:58.316+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:58.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:58.316+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:58.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:58.316+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:58.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:58.316+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:35.262644+0000 front 2024-05-11T12:37:35.262697+0000 (oldest deadline 2024-05-11T12:37:57.562704+0000) 2024-05-11T12:37:58.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:58.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:58.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:58.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:58.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:58.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:58.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:58.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:58.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:58.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:58.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:58 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:58.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:37:58.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:58.466+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:58.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:58.466+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:58.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:58.466+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:58.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:58.466+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:58.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:58.466+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:58.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:58 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:58.466+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:37:59.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:58.861+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:37:59.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:58.861+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:37:59.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:58.861+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:37:59.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:58.861+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:37:59.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:58.861+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:37:59.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:58 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:58.861+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:37:59.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:59.333+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.056478+0000 front 2024-05-11T12:37:01.056670+0000 (oldest deadline 2024-05-11T12:37:26.857259+0000) 2024-05-11T12:37:59.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:59.333+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.857753+0000 front 2024-05-11T12:37:06.857810+0000 (oldest deadline 2024-05-11T12:37:32.157839+0000) 2024-05-11T12:37:59.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:59.333+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.959178+0000 front 2024-05-11T12:37:17.959130+0000 (oldest deadline 2024-05-11T12:37:43.859201+0000) 2024-05-11T12:37:59.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:59.333+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:23.859866+0000 front 2024-05-11T12:37:23.859645+0000 (oldest deadline 2024-05-11T12:37:49.159643+0000) 2024-05-11T12:37:59.614 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8[53422]: 2024-05-11T12:37:59.333+0000 7f7340b3f700 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:35.262644+0000 front 2024-05-11T12:37:35.262697+0000 (oldest deadline 2024-05-11T12:37:57.562704+0000) 2024-05-11T12:37:59.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:59.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:37:59.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:59.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:37:59.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:59.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:37:59.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:59.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:37:59.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:59.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:37:59.615 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:37:59 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:37:59.554+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:37:59.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:59.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:37:59.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:59.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:37:59.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:59.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:37:59.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:59.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:37:59.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:59.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:37:59.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:37:59 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:37:59.464+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:00.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:59.877+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:00.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:59.877+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:00.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:59.877+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:00.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:59.877+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:00.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:59.877+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:00.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:37:59 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:37:59.877+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:00.581 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:00 smithi064 podman[70677]: 2024-05-11 12:38:00.289557761 +0000 UTC m=+5.450825959 container died c2c6f444a384e4373c21124b69678fb98011837dd7c335da229ec65c2976ef71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-11T12:38:00.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:00.483+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:00.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:00.483+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:00.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:00.483+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:00.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:00.483+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:00.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:00.483+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:00.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:00 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:00.483+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:00.864 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:00 smithi064 podman[70677]: 2024-05-11 12:38:00.8059546 +0000 UTC m=+5.967222789 container cleanup c2c6f444a384e4373c21124b69678fb98011837dd7c335da229ec65c2976ef71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:38:00.864 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:00 smithi064 bash[70677]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8 2024-05-11T12:38:00.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:00.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:00.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:00.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:00.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:00.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:00.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:00.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:00.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:00.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:00.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:00 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:00.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:01.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:00.830+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:01.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:00.830+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:01.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:00.830+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:01.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:00.830+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:01.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:00.830+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:01.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:00 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:00.830+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:01.364 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:01 smithi064 podman[70695]: 2024-05-11 12:38:01.049024431 +0000 UTC m=+0.752025139 container remove c2c6f444a384e4373c21124b69678fb98011837dd7c335da229ec65c2976ef71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-11T12:38:01.669 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:01 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:01.630+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:01.669 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:01 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:01.630+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:01.669 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:01 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:01.630+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:01.669 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:01 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:01.630+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:01.669 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:01 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:01.630+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:01.669 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:01 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:01.630+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:01.670 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:01 smithi064 podman[70781]: 2024-05-11 12:38:01.290441091 +0000 UTC m=+0.018713688 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:38:01.670 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:01 smithi064 podman[70781]: 2024-05-11 12:38:01.42499659 +0000 UTC m=+0.153269205 container create 02f764adc7a065c3c5a289d0e967026c35936c7e1b76fce8df5b31012304023f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-11T12:38:01.670 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:01 smithi064 podman[70781]: 2024-05-11 12:38:01.669579309 +0000 UTC m=+0.397851887 container init 02f764adc7a065c3c5a289d0e967026c35936c7e1b76fce8df5b31012304023f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:38:01.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:01 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:01.458+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:01.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:01 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:01.458+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:01.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:01 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:01.458+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:01.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:01 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:01.458+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:01.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:01 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:01.458+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:01.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:01 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:01.458+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:01.984 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:01 smithi064 podman[70781]: 2024-05-11 12:38:01.673069987 +0000 UTC m=+0.401342566 container start 02f764adc7a065c3c5a289d0e967026c35936c7e1b76fce8df5b31012304023f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-11T12:38:01.984 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:01 smithi064 podman[70781]: 2024-05-11 12:38:01.752933365 +0000 UTC m=+0.481205948 container attach 02f764adc7a065c3c5a289d0e967026c35936c7e1b76fce8df5b31012304023f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:38:02.100 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:01.841+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:02.100 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:01.841+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:02.100 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:01.841+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:02.101 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:01.841+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:02.101 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:01.841+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:02.101 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:01 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:01.841+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:02.289 INFO:journalctl@ceph.osd.8.smithi064.stdout:May 11 12:38:01 smithi064 podman[70781]: 2024-05-11 12:38:01.985112719 +0000 UTC m=+0.713385305 container died 02f764adc7a065c3c5a289d0e967026c35936c7e1b76fce8df5b31012304023f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:38:02.631 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.8.service' 2024-05-11T12:38:02.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:02.672 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-11T12:38:02.672 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-11T12:38:02.672 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.9 2024-05-11T12:38:02.702 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:02 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.443+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:02.702 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:02 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.443+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:02.702 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:02 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.443+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:02.702 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:02 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.443+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:02.702 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:02 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.443+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:02.703 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:02 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.443+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:02.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:02.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:02.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:02.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:02.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:02.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:02.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:02.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:02.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:02.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:02.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:02.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:03.000 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:02 smithi047 systemd[1]: Stopping Ceph osd.9 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:03.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:02.850+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:03.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:02.850+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:03.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:02.850+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:03.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:02.850+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:03.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:02.850+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:03.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:02 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:02.850+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:03.275 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.999+0000 7f229a184700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:03.275 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.999+0000 7f229a184700 -1 osd.9 86 *** Got signal Terminated *** 2024-05-11T12:38:03.275 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:02.999+0000 7f229a184700 -1 osd.9 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:38:03.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:03.447+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:03.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:03.447+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:03.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:03.447+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:03.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:03.447+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:03.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:03.447+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:03.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:03 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:03.447+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:03.876 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:03 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:03.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:03.876 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:03 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:03.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:03.877 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:03 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:03.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:03.877 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:03 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:03.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:03.877 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:03 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:03.615+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:03.877 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:03 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:03.616+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:04.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:03.812+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:04.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:03.812+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:04.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:03.812+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:04.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:03.812+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:04.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:03.812+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:04.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:03 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:03.812+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:04.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:04 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:04.411+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:04.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:04 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:04.411+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:04.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:04 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:04.411+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:04.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:04 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:04.411+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:04.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:04 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:04.411+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:04.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:04 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:04.411+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:04.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:04 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:04.588+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:04.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:04 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:04.588+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:04.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:04 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:04.588+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:04.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:04 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:04.588+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:04.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:04 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:04.588+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:04.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:04 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:04.588+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:05.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:04 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:04.856+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:05.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:04 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:04.856+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:05.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:04 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:04.856+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:05.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:04 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:04.856+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:05.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:04 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:04.856+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:05.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:04 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:04.856+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:05.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:04 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:04.856+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:05.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:05 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:05.422+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:05.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:05 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:05.422+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:05.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:05 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:05.422+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:05.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:05 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:05.422+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:05.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:05 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:05.422+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:05.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:05 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:05.422+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:05.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:05.586+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:05.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:05.586+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:05.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:05.586+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:05.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:05.586+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:05.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:05.586+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:05.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:05.586+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:06.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:05 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:05.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:06.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:05 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:05.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:06.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:05 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:05.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:06.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:05 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:05.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:06.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:05 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:05.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:06.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:05 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:05.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:06.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:05 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:05.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:06.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:06.469+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:06.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:06.469+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:06.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:06.469+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:06.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:06.469+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:06.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:06.469+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:06.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:06 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:06.469+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:06.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:06 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:06.547+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:06.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:06 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:06.547+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:06.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:06 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:06.547+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:06.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:06 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:06.547+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:06.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:06 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:06.547+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:06.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:06 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:06.547+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:07.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:06 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:06.901+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:07.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:06 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:06.901+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:07.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:06 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:06.901+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:07.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:06 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:06.901+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:07.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:06 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:06.901+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:07.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:06 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:06.901+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:07.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:06 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:06.901+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:07.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:07.430+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.902075+0000 front 2024-05-11T12:36:49.902046+0000 (oldest deadline 2024-05-11T12:37:14.002210+0000) 2024-05-11T12:38:07.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:07.430+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.004314+0000 front 2024-05-11T12:37:01.004112+0000 (oldest deadline 2024-05-11T12:37:22.704316+0000) 2024-05-11T12:38:07.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:07.430+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:06.805379+0000 front 2024-05-11T12:37:06.805397+0000 (oldest deadline 2024-05-11T12:37:31.406109+0000) 2024-05-11T12:38:07.775 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:07.430+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:17.509500+0000 front 2024-05-11T12:37:17.509522+0000 (oldest deadline 2024-05-11T12:37:41.609458+0000) 2024-05-11T12:38:07.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:07.430+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.710346+0000 front 2024-05-11T12:37:25.710511+0000 (oldest deadline 2024-05-11T12:37:48.010406+0000) 2024-05-11T12:38:07.776 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:07 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9[63611]: 2024-05-11T12:38:07.430+0000 7f2292770700 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.911463+0000 front 2024-05-11T12:37:33.911448+0000 (oldest deadline 2024-05-11T12:37:58.011352+0000) 2024-05-11T12:38:07.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:07 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:07.503+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:07.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:07 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:07.503+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:07.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:07 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:07.503+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:07.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:07 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:07.503+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:07.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:07 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:07.503+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:07.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:07 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:07.503+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:08.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:07 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:07.874+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:08.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:07 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:07.874+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:08.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:07 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:07.874+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:08.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:07 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:07.874+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:08.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:07 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:07.874+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:08.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:07 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:07.874+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:08.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:07 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:07.874+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:08.858 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:08 smithi047 podman[102453]: 2024-05-11 12:38:08.564099891 +0000 UTC m=+5.689778026 container died 0b59aee42bc7aa2ed1c4c09369dac3c85d9278eb09e84c99c8c3e51370b5f978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:38:08.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:08 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:08.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:08.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:08 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:08.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:08.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:08 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:08.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:08.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:08 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:08.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:08.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:08 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:08.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:08.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:08 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:08.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:09.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:08 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:08.883+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:09.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:08 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:08.883+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:09.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:08 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:08.883+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:09.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:08 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:08.883+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:09.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:08 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:08.883+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:09.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:08 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:08.883+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:09.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:08 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:08.883+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:09.275 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:09 smithi047 podman[102453]: 2024-05-11 12:38:09.024739146 +0000 UTC m=+6.150417280 container cleanup 0b59aee42bc7aa2ed1c4c09369dac3c85d9278eb09e84c99c8c3e51370b5f978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-11T12:38:09.275 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:09 smithi047 bash[102453]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9 2024-05-11T12:38:09.658 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:09 smithi047 podman[102469]: 2024-05-11 12:38:09.317849814 +0000 UTC m=+0.750742945 container remove 0b59aee42bc7aa2ed1c4c09369dac3c85d9278eb09e84c99c8c3e51370b5f978 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-11T12:38:09.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:09.563+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:09.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:09.563+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:09.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:09.563+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:09.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:09.563+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:09.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:09.563+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:09.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:09 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:09.563+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:09.912 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:09 smithi047 podman[102563]: 2024-05-11 12:38:09.562530937 +0000 UTC m=+0.019021389 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:38:09.912 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:09 smithi047 podman[102563]: 2024-05-11 12:38:09.668729008 +0000 UTC m=+0.125219447 container create af3969b81272fd00d2b29d9f68c40c3f8b19ce6e89175cb866f28ffc41347504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:38:09.912 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:09 smithi047 podman[102563]: 2024-05-11 12:38:09.912758658 +0000 UTC m=+0.369249101 container init af3969b81272fd00d2b29d9f68c40c3f8b19ce6e89175cb866f28ffc41347504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, io.buildah.version=1.33.5) 2024-05-11T12:38:10.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:09 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:09.835+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:10.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:09 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:09.835+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:10.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:09 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:09.835+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:10.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:09 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:09.835+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:10.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:09 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:09.835+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:10.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:09 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:09.835+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:10.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:09 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:09.835+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:10.225 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:09 smithi047 podman[102563]: 2024-05-11 12:38:09.916299632 +0000 UTC m=+0.372790065 container start af3969b81272fd00d2b29d9f68c40c3f8b19ce6e89175cb866f28ffc41347504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True) 2024-05-11T12:38:10.225 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:10 smithi047 podman[102563]: 2024-05-11 12:38:10.004318574 +0000 UTC m=+0.460809022 container attach af3969b81272fd00d2b29d9f68c40c3f8b19ce6e89175cb866f28ffc41347504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:38:10.515 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:10 smithi047 podman[102563]: 2024-05-11 12:38:10.226447168 +0000 UTC m=+0.682937628 container died af3969b81272fd00d2b29d9f68c40c3f8b19ce6e89175cb866f28ffc41347504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-11T12:38:10.773 INFO:journalctl@ceph.osd.9.smithi047.stdout:May 11 12:38:10 smithi047 podman[102589]: 2024-05-11 12:38:10.757610977 +0000 UTC m=+0.525213172 container remove af3969b81272fd00d2b29d9f68c40c3f8b19ce6e89175cb866f28ffc41347504 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-11T12:38:10.778 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.9.service' 2024-05-11T12:38:10.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:10.861 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-11T12:38:10.861 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-11T12:38:10.861 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.10 2024-05-11T12:38:10.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:10.518+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:10.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:10.518+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:10.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:10.518+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:10.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:10.518+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:10.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:10.518+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:10.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:10.518+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:11.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:10.848+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:11.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:10.848+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:11.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:10.848+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:11.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:10.848+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:11.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:10.848+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:11.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:10.848+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:11.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:10.848+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:11.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:10 smithi050 systemd[1]: Stopping Ceph osd.10 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:11.699 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.213+0000 7f34c874c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:11.699 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.213+0000 7f34c874c700 -1 osd.10 86 *** Got signal Terminated *** 2024-05-11T12:38:11.699 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.213+0000 7f34c874c700 -1 osd.10 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:38:11.777 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:11 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:11.517+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:11.777 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:11 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:11.517+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:11.777 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:11 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:11.517+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:11.777 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:11 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:11.517+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:11.777 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:11 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:11.517+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:11.777 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:11 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:11.517+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:12.155 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.892+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:12.155 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.892+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:12.155 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.892+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:12.155 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.892+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:12.155 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.892+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:12.155 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.892+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:12.156 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:11 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:11.892+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:12.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:12 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:12.541+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:12.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:12 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:12.541+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:12.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:12 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:12.541+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:12.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:12 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:12.541+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:12.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:12 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:12.541+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:12.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:12 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:12.541+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:13.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:12 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:12.895+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:13.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:12 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:12.895+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:13.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:12 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:12.895+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:13.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:12 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:12.895+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:13.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:12 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:12.895+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:13.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:12 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:12.895+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:13.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:12 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:12.895+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:13.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:13 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:13.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:13.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:13 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:13.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:13.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:13 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:13.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:13.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:13 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:13.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:13.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:13 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:13.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:13.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:13 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:13.581+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:14.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:13 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:13.852+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:14.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:13 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:13.852+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:14.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:13 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:13.852+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:14.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:13 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:13.852+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:14.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:13 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:13.852+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:14.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:13 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:13.852+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:14.200 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:13 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:13.852+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:14.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:14 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:14.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:14.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:14 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:14.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:14.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:14 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:14.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:14.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:14 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:14.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:14.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:14 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:14.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:14.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:14 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:14.532+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:15.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:14.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:15.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:14.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:15.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:14.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:15.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:14.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:15.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:14.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:15.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:14.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:15.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:14 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:14.900+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:15.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:15.571+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:15.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:15.571+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:15.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:15.571+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:15.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:15.571+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:15.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:15.571+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:15.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:15.571+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:15.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:15.571+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:16.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:15.904+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:52.562535+0000 front 2024-05-11T12:36:52.562238+0000 (oldest deadline 2024-05-11T12:37:14.262224+0000) 2024-05-11T12:38:16.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:15.904+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:00.163206+0000 front 2024-05-11T12:37:00.163241+0000 (oldest deadline 2024-05-11T12:37:22.463429+0000) 2024-05-11T12:38:16.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:15.904+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.966112+0000 front 2024-05-11T12:37:09.966068+0000 (oldest deadline 2024-05-11T12:37:34.065902+0000) 2024-05-11T12:38:16.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:15.904+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:16.867499+0000 front 2024-05-11T12:37:16.867521+0000 (oldest deadline 2024-05-11T12:37:42.167588+0000) 2024-05-11T12:38:16.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:15.904+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:22.168223+0000 front 2024-05-11T12:37:22.168247+0000 (oldest deadline 2024-05-11T12:37:51.568964+0000) 2024-05-11T12:38:16.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:15.904+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:33.870105+0000 front 2024-05-11T12:37:33.870052+0000 (oldest deadline 2024-05-11T12:37:57.369959+0000) 2024-05-11T12:38:16.199 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:15 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10[59240]: 2024-05-11T12:38:15.904+0000 7f34c0d38700 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.47:6822 osd.6 since back 2024-05-11T12:37:40.171733+0000 front 2024-05-11T12:37:40.171751+0000 (oldest deadline 2024-05-11T12:38:04.271704+0000) 2024-05-11T12:38:16.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:16.556+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:16.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:16.556+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:16.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:16.556+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:16.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:16.556+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:16.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:16.556+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:16.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:16.556+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:16.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:16 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:16.556+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:16.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:16 smithi050 podman[69865]: 2024-05-11 12:38:16.590727534 +0000 UTC m=+5.512285978 container died e7f729a2edcfbe47a86c752ea09879af5c8fe6dbed54e7ae175234babd6f7fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-05-11T12:38:17.306 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 podman[69865]: 2024-05-11 12:38:17.04671375 +0000 UTC m=+5.968272191 container cleanup e7f729a2edcfbe47a86c752ea09879af5c8fe6dbed54e7ae175234babd6f7fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-11T12:38:17.306 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 bash[69865]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10 2024-05-11T12:38:17.669 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 podman[69884]: 2024-05-11 12:38:17.306068217 +0000 UTC m=+0.709879400 container remove e7f729a2edcfbe47a86c752ea09879af5c8fe6dbed54e7ae175234babd6f7fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-11T12:38:17.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:17.560+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:17.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:17.560+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:17.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:17.560+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:17.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:17.560+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:17.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:17.560+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:17.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:17.560+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:17.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:17 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:17.560+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:17.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 podman[69973]: 2024-05-11 12:38:17.573392516 +0000 UTC m=+0.018614714 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:38:17.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 podman[69973]: 2024-05-11 12:38:17.690170567 +0000 UTC m=+0.135392792 container create cc8496cefbade1b99cae2f695f2a7828925f442aced321c6ec84d49470a02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-11T12:38:17.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 podman[69973]: 2024-05-11 12:38:17.875746237 +0000 UTC m=+0.320968420 container init cc8496cefbade1b99cae2f695f2a7828925f442aced321c6ec84d49470a02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-11T12:38:17.949 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 podman[69973]: 2024-05-11 12:38:17.8797249 +0000 UTC m=+0.324947091 container start cc8496cefbade1b99cae2f695f2a7828925f442aced321c6ec84d49470a02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-11T12:38:18.449 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:17 smithi050 podman[69973]: 2024-05-11 12:38:17.951780912 +0000 UTC m=+0.397003105 container attach cc8496cefbade1b99cae2f695f2a7828925f442aced321c6ec84d49470a02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507) 2024-05-11T12:38:18.449 INFO:journalctl@ceph.osd.10.smithi050.stdout:May 11 12:38:18 smithi050 podman[69973]: 2024-05-11 12:38:18.193216232 +0000 UTC m=+0.638438415 container died cc8496cefbade1b99cae2f695f2a7828925f442aced321c6ec84d49470a02d96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-10-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-11T12:38:18.758 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.10.service' 2024-05-11T12:38:18.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:18.806 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-11T12:38:18.806 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-11T12:38:18.806 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.11 2024-05-11T12:38:18.835 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:18.537+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:18.835 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:18.537+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:18.835 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:18.537+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:18.836 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:18.537+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:18.836 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:18.537+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:18.836 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:18.537+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:18.836 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:18.537+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:19.114 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:18 smithi064 systemd[1]: Stopping Ceph osd.11 for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:19.500 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.115+0000 7fbdce87e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:19.500 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.115+0000 7fbdce87e700 -1 osd.11 86 *** Got signal Terminated *** 2024-05-11T12:38:19.500 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.115+0000 7fbdce87e700 -1 osd.11 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-11T12:38:19.500 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.499+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:19.500 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.499+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:19.501 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.499+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:19.501 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.499+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:19.501 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.499+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:19.501 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.499+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:20.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:19 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:19.499+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:20.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:20.458+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:20.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:20.458+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:20.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:20.458+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:20.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:20.458+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:20.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:20.458+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:20.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:20.458+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:20.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:20 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:20.458+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:21.817 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:21.412+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:21.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:21.412+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:21.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:21.412+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:21.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:21.412+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:21.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:21.412+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:21.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:21.412+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:21.818 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:21 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:21.412+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:22.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:22.369+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:22.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:22.369+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:22.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:22.369+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:22.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:22.369+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:22.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:22.369+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:22.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:22.369+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:22.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:22 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:22.369+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:23.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6806 osd.0 since back 2024-05-11T12:36:49.401843+0000 front 2024-05-11T12:36:49.401740+0000 (oldest deadline 2024-05-11T12:37:15.301719+0000) 2024-05-11T12:38:23.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6804 osd.1 since back 2024-05-11T12:37:01.203016+0000 front 2024-05-11T12:37:01.203074+0000 (oldest deadline 2024-05-11T12:37:24.103251+0000) 2024-05-11T12:38:23.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6804 osd.2 since back 2024-05-11T12:37:09.805644+0000 front 2024-05-11T12:37:09.805811+0000 (oldest deadline 2024-05-11T12:37:35.105855+0000) 2024-05-11T12:38:23.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.47:6814 osd.3 since back 2024-05-11T12:37:18.006946+0000 front 2024-05-11T12:37:18.006800+0000 (oldest deadline 2024-05-11T12:37:40.306857+0000) 2024-05-11T12:38:23.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6812 osd.4 since back 2024-05-11T12:37:25.008215+0000 front 2024-05-11T12:37:25.008162+0000 (oldest deadline 2024-05-11T12:37:48.508053+0000) 2024-05-11T12:38:23.864 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6812 osd.5 since back 2024-05-11T12:37:32.509883+0000 front 2024-05-11T12:37:32.509830+0000 (oldest deadline 2024-05-11T12:37:57.210054+0000) 2024-05-11T12:38:23.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.50:6820 osd.7 since back 2024-05-11T12:37:50.012184+0000 front 2024-05-11T12:37:50.012308+0000 (oldest deadline 2024-05-11T12:38:15.312124+0000) 2024-05-11T12:38:23.865 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:23 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11[60304]: 2024-05-11T12:38:23.395+0000 7fbdc6e6a700 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.64:6820 osd.8 since back 2024-05-11T12:37:55.312708+0000 front 2024-05-11T12:37:55.312693+0000 (oldest deadline 2024-05-11T12:38:22.913100+0000) 2024-05-11T12:38:24.934 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:24 smithi064 podman[70898]: 2024-05-11 12:38:24.663083102 +0000 UTC m=+5.651776549 container died 40a6cd66bf73278f1fb72171d2ad7bf178c0d0aa684574dc1d0d21089b4ce2db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:38:25.284 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 podman[70898]: 2024-05-11 12:38:25.008834638 +0000 UTC m=+5.997528082 container cleanup 40a6cd66bf73278f1fb72171d2ad7bf178c0d0aa684574dc1d0d21089b4ce2db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-11T12:38:25.284 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 bash[70898]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11 2024-05-11T12:38:25.614 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 podman[70917]: 2024-05-11 12:38:25.284886734 +0000 UTC m=+0.622431446 container remove 40a6cd66bf73278f1fb72171d2ad7bf178c0d0aa684574dc1d0d21089b4ce2db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-11T12:38:25.891 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 podman[71013]: 2024-05-11 12:38:25.542990123 +0000 UTC m=+0.018290635 image pull 3f6a7beffd02ede0b29737f447d6dc0e4b05cccafe6e95df1ef80c218c7d6305 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8 2024-05-11T12:38:25.891 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 podman[71013]: 2024-05-11 12:38:25.652329741 +0000 UTC m=+0.127630283 container create 909a6d505035fd3440988bae04b372381dfe7efbf83c50241551d9b8e43cc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-11T12:38:25.892 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 podman[71013]: 2024-05-11 12:38:25.888563751 +0000 UTC m=+0.363864260 container init 909a6d505035fd3440988bae04b372381dfe7efbf83c50241551d9b8e43cc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-05-11T12:38:25.892 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 podman[71013]: 2024-05-11 12:38:25.89210459 +0000 UTC m=+0.367405091 container start 909a6d505035fd3440988bae04b372381dfe7efbf83c50241551d9b8e43cc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-11T12:38:26.192 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:25 smithi064 podman[71013]: 2024-05-11 12:38:25.97971811 +0000 UTC m=+0.455018646 container attach 909a6d505035fd3440988bae04b372381dfe7efbf83c50241551d9b8e43cc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:38:26.490 INFO:journalctl@ceph.osd.11.smithi064.stdout:May 11 12:38:26 smithi064 podman[71013]: 2024-05-11 12:38:26.193495302 +0000 UTC m=+0.668795805 container died 909a6d505035fd3440988bae04b372381dfe7efbf83c50241551d9b8e43cc0fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:38:26.781 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@osd.11.service' 2024-05-11T12:38:26.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:26.863 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-11T12:38:26.863 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-11T12:38:26.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.a 2024-05-11T12:38:27.168 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:38:26 smithi047 systemd[1]: Stopping Ceph mds.a for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:27.448 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:38:27 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a[71029]: 2024-05-11T12:38:27.166+0000 7f887ccef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:27.448 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:38:27 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a[71029]: 2024-05-11T12:38:27.166+0000 7f887ccef700 -1 mds.a *** got signal Terminated *** 2024-05-11T12:38:31.502 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:38:31 smithi047 podman[102669]: 2024-05-11 12:38:31.199365671 +0000 UTC m=+4.126973722 container died 1790197da40d80370a7659a871d59b28081aa655e16c159451464b2710867c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:38:31.775 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:38:31 smithi047 podman[102669]: 2024-05-11 12:38:31.667141568 +0000 UTC m=+4.594749665 container cleanup 1790197da40d80370a7659a871d59b28081aa655e16c159451464b2710867c80 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, maintainer=Guillaume Abrioux ) 2024-05-11T12:38:31.775 INFO:journalctl@ceph.mds.a.smithi047.stdout:May 11 12:38:31 smithi047 bash[102669]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-a 2024-05-11T12:38:32.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.a.service' 2024-05-11T12:38:32.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:32.132 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-11T12:38:32.132 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-11T12:38:32.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.d 2024-05-11T12:38:32.275 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:38:32 smithi047 systemd[1]: Stopping Ceph mds.d for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:32.775 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:38:32 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d[69671]: 2024-05-11T12:38:32.463+0000 7fd3446cc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:32.775 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:38:32 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d[69671]: 2024-05-11T12:38:32.463+0000 7fd3446cc700 -1 mds.d *** got signal Terminated *** 2024-05-11T12:38:36.363 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:38:36 smithi047 podman[102801]: 2024-05-11 12:38:36.087957902 +0000 UTC m=+3.729704548 container died 060ade8db75e7d2f6d685e4ac688ff891e6617693bddfb1b6bd729abc08c7c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-11T12:38:36.775 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:38:36 smithi047 podman[102801]: 2024-05-11 12:38:36.554825758 +0000 UTC m=+4.196572439 container cleanup 060ade8db75e7d2f6d685e4ac688ff891e6617693bddfb1b6bd729abc08c7c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-11T12:38:36.775 INFO:journalctl@ceph.mds.d.smithi047.stdout:May 11 12:38:36 smithi047 bash[102801]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-d 2024-05-11T12:38:36.942 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.d.service' 2024-05-11T12:38:36.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:36.977 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-11T12:38:36.977 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-11T12:38:36.977 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.g 2024-05-11T12:38:37.275 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:38:37 smithi047 systemd[1]: Stopping Ceph mds.g for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:37.775 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:38:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g[70599]: 2024-05-11T12:38:37.333+0000 7fc0c5218700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:37.775 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:38:37 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g[70599]: 2024-05-11T12:38:37.333+0000 7fc0c5218700 -1 mds.g *** got signal Terminated *** 2024-05-11T12:38:39.917 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:38:39 smithi047 podman[102920]: 2024-05-11 12:38:39.639404358 +0000 UTC m=+2.433692778 container died 573e378133e2ce8945570bd74778100a7f2784bd8e15b1e00d2edbb9f93a85de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-11T12:38:40.275 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:38:40 smithi047 podman[102920]: 2024-05-11 12:38:40.108768566 +0000 UTC m=+2.903056990 container cleanup 573e378133e2ce8945570bd74778100a7f2784bd8e15b1e00d2edbb9f93a85de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-11T12:38:40.275 INFO:journalctl@ceph.mds.g.smithi047.stdout:May 11 12:38:40 smithi047 bash[102920]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-g 2024-05-11T12:38:40.503 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.g.service' 2024-05-11T12:38:40.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:40.537 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-11T12:38:40.537 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-11T12:38:40.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.j 2024-05-11T12:38:40.775 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:38:40 smithi047 systemd[1]: Stopping Ceph mds.j for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:41.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:38:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j[69338]: 2024-05-11T12:38:40.878+0000 7fab0bb5b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:41.275 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:38:40 smithi047 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j[69338]: 2024-05-11T12:38:40.878+0000 7fab0bb5b700 -1 mds.j *** got signal Terminated *** 2024-05-11T12:38:45.148 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:38:44 smithi047 podman[103041]: 2024-05-11 12:38:44.894139801 +0000 UTC m=+4.135983155 container died f44a52a5393e311676ddeebe3ce09c4b80b6b4ffb4860f87bb24b7546a5928cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:38:45.507 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:38:45 smithi047 podman[103041]: 2024-05-11 12:38:45.331056114 +0000 UTC m=+4.572899514 container cleanup f44a52a5393e311676ddeebe3ce09c4b80b6b4ffb4860f87bb24b7546a5928cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-11T12:38:45.507 INFO:journalctl@ceph.mds.j.smithi047.stdout:May 11 12:38:45 smithi047 bash[103041]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-j 2024-05-11T12:38:45.721 DEBUG:teuthology.orchestra.run.smithi047:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.j.service' 2024-05-11T12:38:45.757 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:45.757 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-11T12:38:45.758 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-11T12:38:45.759 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.b 2024-05-11T12:38:46.049 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:38:45 smithi050 systemd[1]: Stopping Ceph mds.b for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:46.449 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:38:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b[64474]: 2024-05-11T12:38:46.048+0000 7f4b82ba0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:46.449 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:38:46 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b[64474]: 2024-05-11T12:38:46.048+0000 7f4b82ba0700 -1 mds.b *** got signal Terminated *** 2024-05-11T12:38:50.342 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:38:50 smithi050 podman[70079]: 2024-05-11 12:38:50.058552009 +0000 UTC m=+4.102244218 container died 2ccb6b21283749d4973b32ce72ddbb30a8e7bef42fbddfe7168edc863bbda233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-05-11T12:38:50.699 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:38:50 smithi050 podman[70079]: 2024-05-11 12:38:50.449905363 +0000 UTC m=+4.493597610 container cleanup 2ccb6b21283749d4973b32ce72ddbb30a8e7bef42fbddfe7168edc863bbda233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-11T12:38:50.699 INFO:journalctl@ceph.mds.b.smithi050.stdout:May 11 12:38:50 smithi050 bash[70079]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-b 2024-05-11T12:38:50.888 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.b.service' 2024-05-11T12:38:50.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:50.925 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-11T12:38:50.925 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-11T12:38:50.925 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.e 2024-05-11T12:38:51.254 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:51 smithi050 systemd[1]: Stopping Ceph mds.e for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:51.699 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e[63841]: 2024-05-11T12:38:51.254+0000 7f0bd8f58700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:51.699 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:51 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e[63841]: 2024-05-11T12:38:51.254+0000 7f0bd8f58700 -1 mds.e *** got signal Terminated *** 2024-05-11T12:38:52.949 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:52 smithi050 podman[70211]: 2024-05-11 12:38:52.578642515 +0000 UTC m=+1.433239292 container died 230b1ad531e2f327c1489d769e6dc6854be9bf8d4fc2b3a4d7d03b490cdf95e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e) 2024-05-11T12:38:53.406 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:53 smithi050 podman[70211]: 2024-05-11 12:38:53.038583522 +0000 UTC m=+1.893180307 container cleanup 230b1ad531e2f327c1489d769e6dc6854be9bf8d4fc2b3a4d7d03b490cdf95e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-11T12:38:53.406 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:53 smithi050 bash[70211]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e 2024-05-11T12:38:53.633 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.e.service' 2024-05-11T12:38:53.656 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:53 smithi050 podman[70225]: 2024-05-11 12:38:53.406530653 +0000 UTC m=+0.830442894 container remove 230b1ad531e2f327c1489d769e6dc6854be9bf8d4fc2b3a4d7d03b490cdf95e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.vendor=CentOS) 2024-05-11T12:38:53.656 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:53 smithi050 systemd[1]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.e.service: Deactivated successfully. 2024-05-11T12:38:53.656 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:53 smithi050 systemd[1]: Stopped Ceph mds.e for 9b1a9a48-0f8f-11ef-bc98-c7b262605968. 2024-05-11T12:38:53.657 INFO:journalctl@ceph.mds.e.smithi050.stdout:May 11 12:38:53 smithi050 systemd[1]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.e.service: Consumed 1min 38.513s CPU time. 2024-05-11T12:38:53.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:53.667 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-11T12:38:53.667 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-11T12:38:53.667 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.h 2024-05-11T12:38:53.949 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:38:53 smithi050 systemd[1]: Stopping Ceph mds.h for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:54.449 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:38:54 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h[64152]: 2024-05-11T12:38:54.008+0000 7fe4bcdd3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:54.449 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:38:54 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h[64152]: 2024-05-11T12:38:54.008+0000 7fe4bcdd3700 -1 mds.h *** got signal Terminated *** 2024-05-11T12:38:56.876 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:38:56 smithi050 podman[70331]: 2024-05-11 12:38:56.57426624 +0000 UTC m=+2.681748330 container died 7e9c270d0ce37bb7d202828b28dbfe20b98c514103da2ff76f48a1fcfeaea3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-11T12:38:57.168 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:38:56 smithi050 podman[70331]: 2024-05-11 12:38:56.976193947 +0000 UTC m=+3.083676039 container cleanup 7e9c270d0ce37bb7d202828b28dbfe20b98c514103da2ff76f48a1fcfeaea3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-11T12:38:57.168 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:38:56 smithi050 bash[70331]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h 2024-05-11T12:38:57.418 INFO:journalctl@ceph.mds.h.smithi050.stdout:May 11 12:38:57 smithi050 podman[70345]: 2024-05-11 12:38:57.169003705 +0000 UTC m=+0.588188537 container remove 7e9c270d0ce37bb7d202828b28dbfe20b98c514103da2ff76f48a1fcfeaea3aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-h, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240510.053205-3085068) 2024-05-11T12:38:57.421 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.h.service' 2024-05-11T12:38:57.454 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:38:57.455 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-11T12:38:57.455 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-11T12:38:57.455 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.k 2024-05-11T12:38:57.699 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:38:57 smithi050 systemd[1]: Stopping Ceph mds.k for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:38:58.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:38:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k[63522]: 2024-05-11T12:38:57.771+0000 7fb79576f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:38:58.199 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:38:57 smithi050 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k[63522]: 2024-05-11T12:38:57.771+0000 7fb79576f700 -1 mds.k *** got signal Terminated *** 2024-05-11T12:39:01.404 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:39:01 smithi050 podman[70453]: 2024-05-11 12:39:01.085810685 +0000 UTC m=+3.412616317 container died 77aa021b21a68582af448a1962fc252ddae60f071046683b875df4e4b5cadf61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-11T12:39:01.404 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:39:01 smithi050 podman[70453]: 2024-05-11 12:39:01.405171392 +0000 UTC m=+3.731977022 container cleanup 77aa021b21a68582af448a1962fc252ddae60f071046683b875df4e4b5cadf61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-11T12:39:01.699 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:39:01 smithi050 bash[70453]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k 2024-05-11T12:39:01.699 INFO:journalctl@ceph.mds.k.smithi050.stdout:May 11 12:39:01 smithi050 podman[70469]: 2024-05-11 12:39:01.606148934 +0000 UTC m=+0.517977760 container remove 77aa021b21a68582af448a1962fc252ddae60f071046683b875df4e4b5cadf61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-11T12:39:01.837 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.k.service' 2024-05-11T12:39:01.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:39:01.869 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-11T12:39:01.869 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-11T12:39:01.870 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.c 2024-05-11T12:39:02.200 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:39:01 smithi064 systemd[1]: Stopping Ceph mds.c for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:39:02.481 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:39:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c[65737]: 2024-05-11T12:39:02.199+0000 7f9f30301700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:39:02.481 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:39:02 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c[65737]: 2024-05-11T12:39:02.199+0000 7f9f30301700 -1 mds.c *** got signal Terminated *** 2024-05-11T12:39:04.864 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:39:04 smithi064 podman[71118]: 2024-05-11 12:39:04.547623818 +0000 UTC m=+2.474601908 container stop ace55ea90732ef846b6b7221273d6bbf781127f6acb1bcc3f1ffec28a0451110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-11T12:39:04.864 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:39:04 smithi064 podman[71118]: 2024-05-11 12:39:04.573665993 +0000 UTC m=+2.500644081 container died ace55ea90732ef846b6b7221273d6bbf781127f6acb1bcc3f1ffec28a0451110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-11T12:39:05.362 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:39:05 smithi064 podman[71118]: 2024-05-11 12:39:05.032986781 +0000 UTC m=+2.959964869 container cleanup ace55ea90732ef846b6b7221273d6bbf781127f6acb1bcc3f1ffec28a0451110 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-11T12:39:05.362 INFO:journalctl@ceph.mds.c.smithi064.stdout:May 11 12:39:05 smithi064 bash[71118]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-c 2024-05-11T12:39:05.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.c.service' 2024-05-11T12:39:05.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:39:05.546 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-11T12:39:05.546 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-11T12:39:05.546 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.f 2024-05-11T12:39:06.114 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:39:05 smithi064 systemd[1]: Stopping Ceph mds.f for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:39:06.114 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:39:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f[65428]: 2024-05-11T12:39:05.878+0000 7ff2b81db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:39:06.114 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:39:05 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f[65428]: 2024-05-11T12:39:05.878+0000 7ff2b81db700 -1 mds.f *** got signal Terminated *** 2024-05-11T12:39:09.986 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:39:09 smithi064 podman[71251]: 2024-05-11 12:39:09.689359461 +0000 UTC m=+3.912263614 container died 8014ca27afe382545cd44266e55622e0630a8991ae02b8569800194ca82523fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f, org.label-schema.build-date=20240507, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-11T12:39:09.986 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:39:09 smithi064 podman[71251]: 2024-05-11 12:39:09.986386588 +0000 UTC m=+4.209290738 container cleanup 8014ca27afe382545cd44266e55622e0630a8991ae02b8569800194ca82523fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-11T12:39:10.312 INFO:journalctl@ceph.mds.f.smithi064.stdout:May 11 12:39:09 smithi064 bash[71251]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-f 2024-05-11T12:39:10.467 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.f.service' 2024-05-11T12:39:10.500 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:39:10.500 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-11T12:39:10.500 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-11T12:39:10.501 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.i 2024-05-11T12:39:10.815 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:39:10 smithi064 systemd[1]: Stopping Ceph mds.i for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:39:11.114 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:39:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i[66046]: 2024-05-11T12:39:10.813+0000 7fd7c3cd2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:39:11.114 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:39:10 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i[66046]: 2024-05-11T12:39:10.813+0000 7fd7c3cd2700 -1 mds.i *** got signal Terminated *** 2024-05-11T12:39:14.864 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:39:14 smithi064 podman[71372]: 2024-05-11 12:39:14.488612246 +0000 UTC m=+3.766504713 container died 0ebc0df130aa4762a88fa9af8ffb65d805c6159149df89fb148b6c70fbd5139c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, RELEASE=wip-rishabh-testing-20240510.053205-3085068, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-05-11T12:39:15.223 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:39:14 smithi064 podman[71372]: 2024-05-11 12:39:14.956541582 +0000 UTC m=+4.234434043 container cleanup 0ebc0df130aa4762a88fa9af8ffb65d805c6159149df89fb148b6c70fbd5139c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240510.053205-3085068, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-11T12:39:15.223 INFO:journalctl@ceph.mds.i.smithi064.stdout:May 11 12:39:14 smithi064 bash[71372]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-i 2024-05-11T12:39:15.396 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.i.service' 2024-05-11T12:39:15.428 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:39:15.428 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-11T12:39:15.429 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-11T12:39:15.429 DEBUG:teuthology.orchestra.run.smithi064:> sudo systemctl stop ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.l 2024-05-11T12:39:15.760 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:39:15 smithi064 systemd[1]: Stopping Ceph mds.l for 9b1a9a48-0f8f-11ef-bc98-c7b262605968... 2024-05-11T12:39:16.114 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:39:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l[65121]: 2024-05-11T12:39:15.758+0000 7f83e3542700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-11T12:39:16.114 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:39:15 smithi064 ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l[65121]: 2024-05-11T12:39:15.758+0000 7f83e3542700 -1 mds.l *** got signal Terminated *** 2024-05-11T12:39:18.408 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:39:18 smithi064 podman[71497]: 2024-05-11 12:39:18.121927843 +0000 UTC m=+2.467431413 container died 9be27e948c6c4e2b1953901b0cc1c80bf6e91971f0bf694aa8271ffcb141bcca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l, RELEASE=wip-rishabh-testing-20240510.053205-3085068, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-11T12:39:18.827 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:39:18 smithi064 podman[71497]: 2024-05-11 12:39:18.626861124 +0000 UTC m=+2.972364708 container cleanup 9be27e948c6c4e2b1953901b0cc1c80bf6e91971f0bf694aa8271ffcb141bcca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cef619d5ee0e963dd3b2416922b284edd9058913dd64935f4b4f123e8dbdc5f8, name=ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=26c185c6730a227e8afae88cfa8b303d6e5c470e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240510.053205-3085068, io.buildah.version=1.33.5) 2024-05-11T12:39:18.827 INFO:journalctl@ceph.mds.l.smithi064.stdout:May 11 12:39:18 smithi064 bash[71497]: ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968-mds-l 2024-05-11T12:39:18.967 DEBUG:teuthology.orchestra.run.smithi064:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9b1a9a48-0f8f-11ef-bc98-c7b262605968@mds.l.service' 2024-05-11T12:39:18.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-11T12:39:18.998 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-11T12:39:18.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-11T12:39:19.204 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:39:21.905 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-11T12:39:22.118 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:39:24.743 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 --force --keep-logs 2024-05-11T12:39:24.943 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T12:39:27.448 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:39:27.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:39:27.516 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-11T12:39:27.542 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-11T12:39:27.544 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi047/crash 2024-05-11T12:39:27.545 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash -- . 2024-05-11T12:39:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:tar: /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-11T12:39:27.575 INFO:teuthology.orchestra.run.smithi047.stderr:tar: Error is not recoverable: exiting now 2024-05-11T12:39:27.577 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi050/crash 2024-05-11T12:39:27.578 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash -- . 2024-05-11T12:39:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-11T12:39:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-05-11T12:39:27.624 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi064/crash 2024-05-11T12:39:27.625 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash -- . 2024-05-11T12:39:27.667 INFO:teuthology.orchestra.run.smithi064.stderr:tar: /var/lib/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/crash: Cannot open: No such file or directory 2024-05-11T12:39:27.667 INFO:teuthology.orchestra.run.smithi064.stderr:tar: Error is not recoverable: exiting now 2024-05-11T12:39:27.668 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-11T12:39:27.669 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-11T12:39:27.755 INFO:tasks.cephadm:Compressing logs... 2024-05-11T12:39:27.755 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T12:39:27.757 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T12:39:27.760 DEBUG:teuthology.orchestra.run.smithi064:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-11T12:39:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-11T12:39:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:: No such file or directory 2024-05-11T12:39:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log 2024-05-11T12:39:27.783 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.g.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log: /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.j.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.3.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.0.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.x.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.a.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.6.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr: 87.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.a.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-11T12:39:27.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.d.log 2024-05-11T12:39:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.9.log 2024-05-11T12:39:27.793 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.d.log: /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.j.log.gz 2024-05-11T12:39:27.793 INFO:teuthology.orchestra.run.smithi064.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-05-11T12:39:27.793 INFO:teuthology.orchestra.run.smithi064.stderr: /var/log/ceph/cephadm.log 2024-05-11T12:39:27.793 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log 2024-05-11T12:39:27.793 INFO:teuthology.orchestra.run.smithi064.stderr:gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.2.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.f.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.5.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.c.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.c.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.z.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.i.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.11.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.8.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.l.log 2024-05-11T12:39:27.794 INFO:teuthology.orchestra.run.smithi064.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log: /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.l.log: 94.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.l.log.gz 2024-05-11T12:39:27.798 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-11T12:39:27.798 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-11T12:39:27.798 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log 2024-05-11T12:39:27.801 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose --/var/log/ceph/cephadm.log: /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.7.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.b.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.4.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.1.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.y.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.h.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.10.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.e.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.b.log 2024-05-11T12:39:27.802 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.k.log 2024-05-11T12:39:27.833 INFO:teuthology.orchestra.run.smithi047.stderr: 82.6% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-11T12:39:27.857 INFO:teuthology.orchestra.run.smithi064.stderr: 82.1% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-11T12:39:27.859 INFO:teuthology.orchestra.run.smithi064.stderr: 90.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-11T12:39:27.862 INFO:teuthology.orchestra.run.smithi064.stderr: 87.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-11T12:39:27.885 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.b.log: /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.k.log: 87.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.log.gz 2024-05-11T12:39:27.903 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.f.log.gz 2024-05-11T12:39:27.908 INFO:teuthology.orchestra.run.smithi064.stderr: 93.0% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.z.log.gz 2024-05-11T12:39:27.909 INFO:teuthology.orchestra.run.smithi050.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-11T12:39:27.939 INFO:teuthology.orchestra.run.smithi050.stderr: 82.2% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.cephadm.log.gz 2024-05-11T12:39:27.947 INFO:teuthology.orchestra.run.smithi050.stderr: 90.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-11T12:39:27.983 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-11T12:39:27.983 INFO:teuthology.orchestra.run.smithi064.stderr: 91.1% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-11T12:39:28.019 INFO:teuthology.orchestra.run.smithi047.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-11T12:39:28.034 INFO:teuthology.orchestra.run.smithi047.stderr: 94.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.d.log.gz 2024-05-11T12:39:28.040 INFO:teuthology.orchestra.run.smithi047.stderr: 94.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.a.log.gz 2024-05-11T12:39:28.043 INFO:teuthology.orchestra.run.smithi047.stderr: 94.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.g.log.gz 2024-05-11T12:39:28.077 INFO:teuthology.orchestra.run.smithi050.stderr: 92.9% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.y.log.gz 2024-05-11T12:39:28.094 INFO:teuthology.orchestra.run.smithi050.stderr: 94.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.k.log.gz 2024-05-11T12:39:28.097 INFO:teuthology.orchestra.run.smithi050.stderr: 94.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.b.log.gz 2024-05-11T12:39:28.134 INFO:teuthology.orchestra.run.smithi064.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-11T12:39:28.181 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.i.log.gz 2024-05-11T12:39:28.204 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-volume.log.gz 2024-05-11T12:39:28.215 INFO:teuthology.orchestra.run.smithi047.stderr: 90.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph.audit.log.gz 2024-05-11T12:39:28.348 INFO:teuthology.orchestra.run.smithi064.stderr: 94.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.c.log.gz 2024-05-11T12:39:28.589 INFO:teuthology.orchestra.run.smithi064.stderr: 91.8% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.c.log.gz 2024-05-11T12:39:30.365 INFO:teuthology.orchestra.run.smithi047.stderr: 90.2% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mgr.x.log.gz 2024-05-11T12:39:32.279 INFO:teuthology.orchestra.run.smithi050.stderr: 91.7% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.b.log.gz 2024-05-11T12:39:32.345 INFO:teuthology.orchestra.run.smithi047.stderr: 90.2% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mon.a.log.gz 2024-05-11T12:40:33.262 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.5.log.gz 2024-05-11T12:40:38.991 INFO:teuthology.orchestra.run.smithi047.stderr: 93.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.0.log.gz 2024-05-11T12:41:22.226 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.2.log.gz 2024-05-11T12:41:35.358 INFO:teuthology.orchestra.run.smithi064.stderr: 93.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.8.log.gz 2024-05-11T12:41:41.739 INFO:teuthology.orchestra.run.smithi064.stderr: 93.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.11.log.gz 2024-05-11T12:41:41.741 INFO:teuthology.orchestra.run.smithi064.stderr: 2024-05-11T12:41:41.742 INFO:teuthology.orchestra.run.smithi064.stderr:real 2m13.967s 2024-05-11T12:41:41.742 INFO:teuthology.orchestra.run.smithi064.stderr:user 4m5.496s 2024-05-11T12:41:41.742 INFO:teuthology.orchestra.run.smithi064.stderr:sys 0m8.800s 2024-05-11T12:42:36.068 INFO:teuthology.orchestra.run.smithi047.stderr: 93.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.6.log.gz 2024-05-11T12:43:56.708 INFO:teuthology.orchestra.run.smithi047.stderr: 93.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.9.log.gz 2024-05-11T12:44:17.724 INFO:teuthology.orchestra.run.smithi047.stderr: 93.2% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.3.log.gz 2024-05-11T12:44:17.726 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-05-11T12:44:17.727 INFO:teuthology.orchestra.run.smithi047.stderr:real 4m49.955s 2024-05-11T12:44:17.727 INFO:teuthology.orchestra.run.smithi047.stderr:user 4m13.054s 2024-05-11T12:44:17.727 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m10.065s 2024-05-11T12:48:53.741 INFO:teuthology.orchestra.run.smithi050.stderr: 97.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.e.log.gz 2024-05-11T12:50:25.047 INFO:teuthology.orchestra.run.smithi050.stderr: 93.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.10.log.gz 2024-05-11T12:51:38.511 INFO:teuthology.orchestra.run.smithi050.stderr: 93.3% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.7.log.gz 2024-05-11T12:55:47.115 INFO:teuthology.orchestra.run.smithi050.stderr: 93.5% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.4.log.gz 2024-05-11T12:57:25.748 INFO:teuthology.orchestra.run.smithi050.stderr: 93.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-osd.1.log.gz 2024-05-11T12:59:48.104 INFO:teuthology.orchestra.run.smithi050.stderr: 94.4% -- replaced with /var/log/ceph/9b1a9a48-0f8f-11ef-bc98-c7b262605968/ceph-mds.h.log.gz 2024-05-11T12:59:48.106 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-11T12:59:48.106 INFO:teuthology.orchestra.run.smithi050.stderr:real 20m20.332s 2024-05-11T12:59:48.106 INFO:teuthology.orchestra.run.smithi050.stderr:user 8m10.872s 2024-05-11T12:59:48.106 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m24.734s 2024-05-11T12:59:48.107 INFO:tasks.cephadm:Archiving logs... 2024-05-11T12:59:48.108 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi047/log 2024-05-11T12:59:48.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T13:00:26.086 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi050/log 2024-05-11T13:00:26.087 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T13:01:31.676 DEBUG:teuthology.misc:Transferring archived files from smithi064:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi064/log 2024-05-11T13:01:31.677 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-11T13:02:10.071 INFO:tasks.cephadm:Removing cluster... 2024-05-11T13:02:10.071 DEBUG:teuthology.orchestra.run.smithi047:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 --force 2024-05-11T13:02:10.293 INFO:teuthology.orchestra.run.smithi047.stdout:Deleting cluster with fsid: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T13:02:10.829 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 --force 2024-05-11T13:02:11.190 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T13:02:11.949 DEBUG:teuthology.orchestra.run.smithi064:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9b1a9a48-0f8f-11ef-bc98-c7b262605968 --force 2024-05-11T13:02:12.180 INFO:teuthology.orchestra.run.smithi064.stdout:Deleting cluster with fsid: 9b1a9a48-0f8f-11ef-bc98-c7b262605968 2024-05-11T13:02:12.739 INFO:tasks.cephadm:Removing cephadm ... 2024-05-11T13:02:12.739 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-11T13:02:12.759 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-11T13:02:12.780 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-11T13:02:12.796 INFO:tasks.cephadm:Teardown complete 2024-05-11T13:02:12.796 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-11T13:02:12.807 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-11T13:02:12.807 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T13:02:12.810 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T13:02:12.822 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-11T13:02:12.951 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-11T13:02:12.961 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-11T13:02:12.961 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T13:02:12.964 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T13:02:12.967 DEBUG:teuthology.orchestra.run.smithi064:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-11T13:02:12.980 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-05-11T13:02:12.981 INFO:teuthology.orchestra.run.smithi064.stderr:bash: line 1: ntpq: command not found 2024-05-11T13:02:12.981 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-11T13:02:12.984 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T13:02:12.984 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-05-11T13:02:12.985 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +971us[ +987us] +/- 50ms 2024-05-11T13:02:12.985 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -984us[ -984us] +/- 76ms 2024-05-11T13:02:12.985 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 -567us[ -567us] +/- 77ms 2024-05-11T13:02:12.985 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:02:12.992 INFO:teuthology.orchestra.run.smithi064.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T13:02:12.992 INFO:teuthology.orchestra.run.smithi064.stdout:=============================================================================== 2024-05-11T13:02:12.992 INFO:teuthology.orchestra.run.smithi064.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 +977us[ +972us] +/- 50ms 2024-05-11T13:02:12.992 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -975us[ -980us] +/- 76ms 2024-05-11T13:02:12.992 INFO:teuthology.orchestra.run.smithi064.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -554us[ -559us] +/- 77ms 2024-05-11T13:02:12.993 INFO:teuthology.orchestra.run.smithi064.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:02:13.034 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-11T13:02:13.034 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-11T13:02:13.035 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +934us[ +914us] +/- 50ms 2024-05-11T13:02:13.035 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -1021us[-1021us] +/- 76ms 2024-05-11T13:02:13.035 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 -580us[ -600us] +/- 77ms 2024-05-11T13:02:13.035 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-11T13:02:13.036 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-11T13:02:13.045 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-11T13:02:13.046 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-11T13:02:13.055 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T13:02:13.185 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T13:02:13.280 DEBUG:teuthology.orchestra.run.smithi064:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-11T13:02:13.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T13:02:13.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:02:13.411 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T13:02:13.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:02:13.443 DEBUG:teuthology.orchestra.run.smithi064:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-11T13:02:13.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:02:13.471 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-11T13:02:13.481 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-11T13:02:13.490 INFO:teuthology.task.internal:Duration was 3422.075326 seconds 2024-05-11T13:02:13.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-11T13:02:13.505 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-11T13:02:13.505 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T13:02:13.507 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T13:02:13.510 DEBUG:teuthology.orchestra.run.smithi064:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-11T13:02:13.563 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T13:02:13.566 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T13:02:13.587 INFO:teuthology.orchestra.run.smithi064.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-11T13:02:14.000 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-11T13:02:14.001 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-05-11T13:02:14.001 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T13:02:14.027 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-05-11T13:02:14.027 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T13:02:14.058 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi064.front.sepia.ceph.com 2024-05-11T13:02:14.058 DEBUG:teuthology.orchestra.run.smithi064:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-11T13:02:14.099 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-11T13:02:14.099 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T13:02:14.101 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T13:02:14.104 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-11T13:02:14.177 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-11T13:02:14.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T13:02:14.180 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T13:02:14.183 DEBUG:teuthology.orchestra.run.smithi064:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-11T13:02:15.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-11T13:02:15.029 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-11T13:02:15.029 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T13:02:15.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T13:02:15.084 DEBUG:teuthology.orchestra.run.smithi064:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-11T13:02:15.113 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-11T13:02:15.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T13:02:15.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T13:02:15.130 DEBUG:teuthology.orchestra.run.smithi064:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-11T13:02:15.156 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-05-11T13:02:15.162 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-05-11T13:02:15.193 INFO:teuthology.orchestra.run.smithi064.stdout:kernel.core_pattern = core 2024-05-11T13:02:15.225 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T13:02:15.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:02:15.240 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T13:02:15.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:02:15.256 DEBUG:teuthology.orchestra.run.smithi064:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-11T13:02:15.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-11T13:02:15.282 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-11T13:02:15.293 INFO:teuthology.task.internal:Transferring archived files... 2024-05-11T13:02:15.294 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi047 2024-05-11T13:02:15.294 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T13:02:15.351 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi050 2024-05-11T13:02:15.351 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T13:02:15.389 DEBUG:teuthology.misc:Transferring archived files from smithi064:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-05-11_05:57:22-fs-wip-rishabh-testing-20240510.053205-testing-default-smithi/7702358/remote/smithi064 2024-05-11T13:02:15.390 DEBUG:teuthology.orchestra.run.smithi064:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-11T13:02:15.431 INFO:teuthology.task.internal:Removing archive directory... 2024-05-11T13:02:15.431 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T13:02:15.433 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T13:02:15.436 DEBUG:teuthology.orchestra.run.smithi064:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-11T13:02:15.482 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-11T13:02:15.494 INFO:teuthology.task.internal:Not uploading archives. 2024-05-11T13:02:15.494 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-11T13:02:15.505 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-11T13:02:15.505 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T13:02:15.508 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T13:02:15.510 DEBUG:teuthology.orchestra.run.smithi064:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-11T13:02:15.521 INFO:teuthology.orchestra.run.smithi047.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 13:02 /home/ubuntu/cephtest 2024-05-11T13:02:15.524 INFO:teuthology.orchestra.run.smithi050.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 13:02 /home/ubuntu/cephtest 2024-05-11T13:02:15.538 INFO:teuthology.orchestra.run.smithi064.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 11 13:02 /home/ubuntu/cephtest 2024-05-11T13:02:15.539 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-11T13:02:15.550 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-11T13:02:15.575 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} duration: 3422.0753257274628 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-05-11T13:02:15.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-11T13:02:15.662 INFO:teuthology.run:pass